Welcome to the comprehensive repository for DSCI 510 - Foundations of Data Management at USC during the Spring 2023 semester, taught by Professor Wensheng Wu. This centralized hub contains all coursework materials, including assignments and project solutions, organized into folders representing distinct modules covered in the course.
Within this repository, assignments and projects delve into fundamental concepts like data storage, file systems, relational vs. semi-structured data, SQL, and more. The course explores big data analytics, addressing NoSQL databases, cloud storage, distributed file systems, and MapReduce. Advanced topics like data cleaning, transformation, warehousing, and integration are also covered. It's essential to use this repository responsibly, refraining from engaging in plagiarism-related activities.
Tip
Before exploring the materials, take a moment to review the license and disclaimer for responsible utilization. The repository covers various topics, providing valuable insights and hands-on experience in Data Management.
- Course Name: DSCI 551 - Foundations of Data Management
- Instructor: Prof. Wensheng Wu
- Semester: Spring 2023
- About the Course:
- Function and design of modern storage systems, including cloud; data management techniques; data modeling; network attached storage, clusters, and data centers; relational databases; the map-reduce paradigm.
Feel free to explore the assignments, projects, and solutions provided as learning aids. Whether you're a beginner or an experienced practitioner, this repository aims to be your companion in mastering the intersection of foundational data management fundamentals within Data Science & Engineering. Happy learning!
Caution
Please note that this repository serves as a reference guide and should be utilized as a tool for learning and comprehension. It's paramount to refrain from engaging in any activities associated with plagiarism. Embrace the wealth of knowledge herein to enhance your understanding and augment your skill set in Data Engineering.
| Assignment | Topic Covered |
|---|---|
| Homework 1 | Analyse Air Quality Index (AQI) data with Firebase Realtime DB |
| Homework 2 | Emulate HDFS command line using Python |
| Homework 3 | Understanding the usage of SQL and XML parsing on HDFS snapshot as XML document |
| Homework 4 | Indexing and Querying on B+ Trees |
| Homework 5 | Analysing AQI Index data of multiple countries using Hadoop MapReduce & Spark |
| --- | --- |
| Lab 1 | Playing around with file system using the bash command line on AWS EC2 |
| Lab 2 | Setup HDFS on AWS EC2 and execute basic file system commands |
| Lab 3 | Ingest and Query AQI data on MongoDB using Mongo Atlas |
| Lab 4 | Analyse Books data on AWS DynamoDB |
| --- | --- |
| Exam Solutions | Consists of PDF documents with snapshots of 2 midterms and final exam quizzes along with solutions |
| --- | --- |
| Project Documents | Consists of all documentation related to the development of the final project |
| Project Repository | Firebase Realtime Database Emulator using FastAPI and MongoDB |
- Kayvan Shah |
MS in Applied Data Science|University of Southern California
This repository is licensed under the BSD 5-Clause License. See the LICENSE file for details.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent