DE4E: Data Engineering for Everybody by Pseudo-Lab.
Loved the project? Please visit our Website
Welcome to our DE4E repository! We aim to give you a complete understanding of data engineering, from fundamentals to advanced concepts. Whether you're new or experienced, our repository empowers data lovers with the knowledge and skills for success in the data-driven era. Join us on this exciting journey as we unlock the full potential of data engineering together!
click the above image will guide you to DE4E website
β‘οΈ move to the website: pseudo-lab.github.io/data-engineering-for-everybody
DE4E: Data Engineering for Everybodyλ κ°μ§μ°κ΅¬μμ DSF νλ‘κ·Έλ¨μμ μμλμμ΅λλ€.
μμμ μμ κ°μ¬μ λ§μμ μ ν©λλ€.
κ°μ§μ°κ΅¬μλ DataCampμ νμμ λ°μ Donates νλ‘κ·Έλ¨μ μ§ννκ³ μμ΅λλ€. νλ‘κ·Έλ¨μ ν΅ν΄ ꡬμ§μ, λΆμμ μ·¨μ μ, λΉμ리 μ°κ΅¬ κ³Όνμ, νμλΆλ€κ» DataCampμμ μ 곡νλ λ€μν μ½μ€μ νΈλμ μ 곡ν©λλ€. λ³Έ νλ‘μ νΈλ DataCamp Donates νλ‘κ·Έλ¨ μ€ νλμΈ Data Science FellowshipμΌλ‘λΆν° μμλμμ΅λλ€.
DE4Eλ λ°μ΄ν° λΆμκ°, λ°μ΄ν° κ³Όνμ, λ°μ΄ν° μμ§λμ΄, λ¨Έμ λ¬λ μμ§λμ΄κ° ν¨κ» λͺ¨μ¬ λ°μ΄ν°μ, λ°μ΄ν°μ μν, λ°μ΄ν°λ₯Ό μν Data Engineering Repositoryλ₯Ό λ§λ€μ΄ λκ°κ³ μ ν©λλ€.
- Self-Check List
- Session 1. Introduction to Data Engineering
- Session 2. Data Sources and Data Collection
- Session 3. Data Transformation and Cleaning
- Session 4. Data Storage
- Session 5. Data Processing Frameworks
- Session 6. Data Processing Frameworks II
- Session 7. Introduction to Apache Airflow
- Session 8. Cloud Computing and Data Engineering
- Capstone Project(In Progress)
| idx | Date | Subject | Presenter | Pre-Question | Tag |
|---|---|---|---|---|---|
| 0 | 2023-03-26 | Session 0. Orientation | μ΄μμ | Why should we learn Data Engineering? | #OT #Direction # Motivation |
| 1 | 2023-04-02 | Session 1. Introduction to Data Engineering | μ΄μμ | What is Data Engineering? | #Data Engineering #Discussion |
| 2 | 2023-04-09 | Session 2. Data Sources and Data Collection | μ΄λμ±, κΉμΈν | How can we collect data from variaty sources? | #Source Data #Data Collection #Data Type #Structured Data #Unstructured Data #Batch Data #Real-time Data |
| 3 | 2023-04-16 | Session 3. Data Transformation and Cleaning | μ΄μμ | How can we transform data more efficiently? | #Data Processing |
| 4 | 2023-04-30 | Session 4. Data Storage | μ‘μ€νΈ, μ ν¬μ | How can we store data more efficiently? | #Data Store #Database #Data Lake #Lakehouse #Object-Storage #NoSQL |
| 5 | 2023-05-07 | Session 5. Data Processing Frameworks | μ κ²½λ₯, μ΄νλ¦Ό | How data processing framework help us? | #Hadoop Eco-system #Parallel Computing |
| 6 | 2023-05-14 | Session 6. Data Processing Frameworks II | κΉμμ , μ΅νμΉ | Learn about various data processing framework | #Apache Spark #Apache Kafka #Apache Storm #Apache Flink |
| 7 | 2023-05-28 | Session 7. Introduction to Apache Airflow | κΉμ±ν, μ΄ν¬λ―Ό | How can we schedule, orchestrate data processing? | #Apache Airflow #Tutorial |
| 8 | 2023-06-04 | Session 8. Cloud Computing and Data Engineering | μ΄λ―Όν, μ΄μμ | What is Cloud Computing? and Why it is so important? | #Cloud Computing #Multi-Cloud #Data Engineering |
| 9 | 2023-06-18 | Capstone Project | μ΄νλ¦Ό | Let's dive into Data Engineering Capstone Proeject | #Capstone Project |
| 10 | 2023-07-09 - | Project Management | μ΄μμ , μ ν¬μ , μ κ²½λ₯, μ΄λμ±, κΉμμ | Build Together! | #Share #Motivation #Delighted to work together #Pseudo-Lab |
κ°μ§μ°κ΅¬μλ λ¨Έμ λ¬λ, λ°μ΄ν° μ¬μ΄μΈμ€, λ°μ΄ν° μμ§λμ΄λ§μ μ€μ¬μΌλ‘ λͺ¨μΈ λΉμ리λ¨μ²΄μ
λλ€. λꡬλ μνλ μ°κ΅¬λ₯Ό ν μ μλ μμμ μ΄ λλ, μ§μ§λ³΄λ€ λ μ§μ§ κ°μ μ°κ΅¬μλ₯Ό κΏκΎΈκ³ μμ΅λλ€. 곡μ (Share), λκΈ°λΆμ¬(Motivation), ν¨κ»νλ μ¦κ±°μ(Delighted to work together)λΌλ ν΅μ¬κ°μΉλ₯Ό μΆκ΅¬νλ©° μ½ 1800μ¬ λͺ
μ μ°κ΅¬μλΆλ€μ΄ μ€λλ ν¨κ» λ¨Έμ λ¬λ, λ°μ΄ν° μ¬μ΄μΈμ€, λ°μ΄ν° μμ§λμ΄λ§ λΆμΌμ μ ν μν₯λ ₯μ νμ¬νκ³ μμ΅λλ€. λ³΄λ€ μμΈν λ΄μ©μ μ¬κΈ°μ μ΄ν΄λ³΄μ€ μ μμ΅λλ€.
This project is licensed under MIT license.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent