IST 722 - DATA WAREHOUSE

Welcome to the Stack Overflow Dashboard repository! This project represents the collaborative efforts of Group 7 in creating a comprehensive analytics solution for Stack Overflow data. We've performed data profiling, executed ETL jobs on Snowflake, utilized dbt for modeling, and integrated everything into Power BI dashboards.

This project is centered around establishing a comprehensive data warehouse solution tailored for Stack Overflow. Following the development of this foundational data infrastructure, our objective extends to crafting an interactive Power BI dashboard that not only ensures user-friendliness but also captivates with its visually compelling interface.

• Comprehensive data profiling for Stack Overflow datasets.
• Efficient ETL jobs on Snowflake for data extraction, transformation, and loading.
• dbt modeling to structure and organize data for analytical purposes.
• Integration of all processed data into Power BI dashboards for easy visualization.

• Snowflake
• dbt (data build tool)
• Power BI

In the initial phase of our project, our team undertook a comprehensive data profiling initiative on Stack Overflow datasets. This crucial step involved the creation of a business process, including a business matrix, dimensional tables, and fact tables, adhering to the principles of enterprise bus architecture.

The ETL (Extract, Transform, Load) process was performed on Snowflake, a cloud-based data warehousing platform. This involved extracting raw data from CSV / JSON files, transforming it to meet our analytical needs, and loading it into Snowflake for analytical purposes.

dbt was used for modeling the processed data, creating a structured and organized foundation for analytics. This step allowed us to define relationships, aggregations, and business logic that facilitated the development of insightful dashboards.

Lineages for 3 defined business processes -

The final output of our efforts is a set of Power BI dashboards that visualize the key insights derived from the Stack Overflow data. These dashboards are designed to be user-friendly, allowing stakeholders to interact with the data and gain valuable insights.

Amisha Gangwar, Tejas Gawade, Lyrica Dodati