adrianriesco / masters-thesis-on-big-data Goto Github PK
View Code? Open in Web Editor NEWMaster's thesis on Big Data
License: Apache License 2.0
Master's thesis on Big Data
License: Apache License 2.0
Integrate Cassandra with Airflow and Spark using Docker Compose.
Write section 6 "Related works" of the report.
Document the objectives and main goals to be achieved with this project.
Redesign the project plan for May and June
Write the API inspection process in the "Programmer guide" section of the memory.
Update the main document with a brief explanation of the selected tools and the work methodology used in the development.
Write section 5 "Relevant aspects of the project".
Learn the basics of Flask in order to build the front-end.
Complete Appendix D "Programmer's guide" of the report.
Complete Appendix C "Design" of the report.
Configure the DAG in Airflow and ensure it is properly loaded in the Docker container.
Write a description of an orchestrator in the project report.
Write a description of NoSQL Databases in the project report.
Include .out, .aux... into gitignore to avoid these files from including in the commits
Improve and unify the code written to collect data from both APIs.
Set up the Airflow environment using Docker as the tool to deploy the containers.
Write the description of the Spotify data in the annex "C. Design".
Write section 7 "Conclusions of the project".
Review the entire project report.
Deploy Cassandra environment via Docker.
Write the project objectives on the memory.
Complete Appendix E "User manual" of the report.
Add a brief description of the tools selected to the memory of the project.
Complete Appendix A "Project plan" of the report.
Write a description of Spotify API in the memory.
Write a description of Flask, Jinja and Bootstrap in the project report.
Write a description of Apache Spark in the project report.
Write a description of CI/CD in the report.
Complete Appendix B "Requirements" of the report.
Set up the Spark environment using Docker as the tool to deploy the containers.
Configure the project memory template to be aligned with the requirements of the project.
Inspect Twitter API to understand its information and working.
Create the ETL workflow to load to Cassandra the data extracted from Twitter and Spotify and processed by Spark.
Configure the work environment where the project will be developed.
Write the python code to gather information from the Twitter API.
Create the front-end of the project with Flask and Bootstrap.
Integrate the ETL workflow with the front-end.
Write the description of the Twitter data in the annex "C. Design".
Write the python code to gather information from the Spotify API.
Write a description of Docker and Docker Compose in the project report.
Write a description of Twitter API in the memory.
Inspect the API of Spotify to understand the expected input and outputs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.