Awesome dbt

Welcome to the awesome curated list of dbt resources!

Any kind of contribution is greatly encouraged and appreciated. For making a contribution, please check the contribution guidelines first! Add new entries on the top of sections (LIFO) to keep fresh items more visible! Also, feel free to add new sections.

Happy contributing!

Contents

• Get Started
• How To
• Integrations
• User Stories
• Data Quality
• CI/CD
• Orchestration
• Utilities
• Packages
• Community
• Sample Projects
• Contributors

Get Started

Courses from where you can get started with Analytics Engineering.

• Data Engineering Zoomcamp - Data engineering course on cutting edge tools including dbt.
• Analytics Engineering with dbt - Paid course offered by co:rise covering the basics of dbt.
• dbt Fundamentals - Official free course offered by dbt. Excellent for learning the basics of dbt Cloud.
• Refactoring SQL for Modularity - Another dbt labs offered free course on dbt refactoring and CTE supercharging.
• Learn DBT from Scratch - Guides you through a setup paired with Snowflake (decorated with extras).

How To

Helping hand on setting up integrations and implementing best practices.

• How we structure our dbt projects - How the dbt team structures its dbt projects.
• dbt guide - Primer on how you should properly set up and configure your dbt workflow.
• dbt for Data Transformation – Hands-on - Yet another tutorial for using dbt Cloud.
• Start Modeling Data - Configuring Bigquery with your dbt project.
• Accelerating Data Teams with dbt & Snowflake - A dbt & Snowflake workshop on financial data.
• Creating a dev environment quickly on Snowflake - Setting up teh integraton with Snowflake.
• How to set up a dbt data-ops workflow, using dbt cloud and Snowflake - Leverage GitHub Actions to set up CI/CD with dbt Core.
• How to configure your dbt repository - Mono-repo or not mono-repo?
• Best Practices for Optimizing Your dbt and Snowflake Deployment - Pocket guide on optimization best practices with Snowflake.
• How to Deploy dbt to Production using GitHub Actions
• Doing More With Less: Using DBT to load data from AWS S3 to Snowflake via External Tables - An alternative guide to set up your dbt-external-tables workflow.
• Best Practices for your dbt Style Guide - Standards for well organized base layer with Airbyte ingestion.
• Tips and Tricks about working with dbt - Tips from community members.
• Writing Unit Tests for dbt - An overview about the package dbt-unit-testing.

Integrations

Collection of known data integrations with dbt

• Model synchronization from dbt to Metabase.
• Soda - Integration of Soda's data observability platform and dbt.
• Supported Adapters - Offically supported database adapters.
• Lightdash - Open source Looker alternative with deep dbt integration.
• Superset - Open source visualization layer for your Modern Data Stack.
• Dagster and dbt: Better Together - Getting started with the dagster-dbt library.
• fal - Add multi-language support (Python) to your dbt project.

User Stories

Use-cases and user stories implemented by the community members using components of the MDS with dbt.

• Leveraging DBT as a Data Modeling tool - Reflection on one-year usage of dbt.
• dbt + Materialize: Streaming to a dbt project near you - How to own your real-time transformation workflows like batch-based alternatives.
• Who's really using dbt? - Behind the community of analytics engineers.
• dbt and the Analytics Engineer — what's the hype about - Behind the upheaval of the analytics engineer profession.
• Analyzing Fishtown's dbt project performance with artifacts - Using project artifacts to identify anomalies and room for refactoring.
• Deploying and Running dbt on Azure Container Instances - Demonstration of integration with Azure.
• Beware of DBT Incremental Updates Against Snowflake External Tables - Things you should be aware of when using external tables with dbt.
• dbt development at Vimeo - Best practises from the Vimeo Data team.

Data Quality

Best-practices and extensions of the testing framework.

• Environment-dependent Unit Testing in dbt - Guide on how to run unit tests in dbt dynamically.
• dbt-expectations - Port between dbt and great_expectations to extend out-of-the-box tests.
• re_data - A dbt package for montioring metrics and detect anomalies.
• How do you test your data - Suggestions on testing your data powered by the community.
• How to unit test sql transforms in dbt - Unit test using source defer and generic custom tests.

CI/CD

Make the best out of your product quality and seamless delivery.

• Anatomy of A Pipeline: CI/CD For a dbt Data Warehouse on Google Big Query Using Azure Pipelines - Setting up CI/CD for a Big Query Stack using Azure Pipelines.
• Continuous Integration and Automated Build Testing with dbtCloud - Great and detailed blogpost on setting up Slim CI in dbt Cloud.
• How to review an analytics pull request - Checkpoints to consider when reviewing an analytics engineer PR.
• Performing a blue/green deploy of your dbt project on Snowflake - A very tidy and fail-safe way to run dbt in production by using two parallel production enviromnents.
• How we speed up our CI runs by 10x using Slim CI - Limit data in long-running CI checks to improve developing experience.

Orchestration

Resources to manage and maintain dependencies in modern data pipelines.

• Building a Scalable Analytics Architecture with Airflow and dbt - Leveraging the dbt manifest in Airflow.
• Auto-generating an Airflow DAG using the dbt manifest - Yet another article on extracting value from the manifest file.
• Building a robust data pipeline with the dAG stack: dbt, Airflow, Great Expectations - Demonstration of a data orchestration project with Airflow.
• Run dbt in Azure Data Factory - Primer about dbt on Azure Data Stack.

Utilities

Useful tools and extensions to bump up your analytics engineer worklow.

• How we set up our computers for working on dbt projects - Things I wish I would have known when started working with dbt. Tools and hacks to improve developing experience.
• fzf-dbt - Search dbt models interactively from terminal.
• vscode-dbt-power-user - VSCode extension to give more clarity on model dependencies.
• Your Essential dbt Project Checklist - Checklist on items necessary for a successful dbt project.
• dbt Style Guide - Developing styleguide often referred in PR templates.
• Clean your warehouse of old and deprecated models - Clean out warehouse models which are not existent in the project.
• dbt-tips - Excellent companion to your dbt practice with rich collection of tips.
• dbt-tags - Understanding the scopes of dbt tags.
• Pre-commit hooks - Pre-commit hooks for checking data integity before schema change commit.

Packages

Community-developed packages to extend default macros and toolset.

• dbt-coves - CLI tool for generating a scaffold for your dbt project.
• dbt-profiler - Data profiling and doc block generator.
• dbt_utils - General macros library. A must have.
• dbt_audit_helper - Macros for data audits that compare columns values and schemas between tables.
• dbt-ml-preprocessing - A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.
• dbt-external-tables - Macros to stage your external sources.
• dbt-feature-store - Macros to build a feature store right within your dbt project.
• dbt-codegen - Macros that generate dbt code, and log it to the command line.
• dbt-init - Create a project and populate as much of the dbt project as possible.
• dbt-artifacts - This package builds a mart of tables from dbt artifacts loaded into a table.
• dbt-erdiagram-generator - This packages generate ERD diagrams from a dbt project.
• Terraform-dbt Cloud Module - IAC in dbt Cloud via Terraform.
• dbt2looker - Generate Looker views for dbt models.
• dbt-coverage - Checks dbt docs and tests coverage.
• dbt-meta-testing - Yet another coverage testing.
• dbt-superset-lineage - Push and pull metadata between dbt to Superset.
• dbtvault - Package for generating and executing ETL for Data Vault 2.0 on Snowflake.
• dbt-invoke - CLI for creating, updating, and deleting dbt property files.
• dbt-unit-testing - Package which contains macros to support unit testing.

Community

Conferences, meetups, dicussions, newsletters, podcasts, etc. led by fellow analytics engineers and forums of contact.

• Coalesce 2021 - Second iteration of the analytics engineer conference.
• Coalesce 2020 - Annual dbt conference full of fascinating use-cases.
• dbt meetups - List of community led dbt meetups.
• Analytics Engineer Roundup - Official dbt Labs newsletter on topics of the MDS.
• Benn Stacil's Newsletter - Tought-provoking reads from founder of Mode.
• Data Engineering Weekly - Weekly newsletter of recent trends in Data Engineering.
• Data Engineering Podcast - One of the most popular data engineering podcasts covering great concepts and new products.
• Analyitics Engineer Podcast - Official podcast of dbt Labs.
• dbt Slack - Energy-filled hub of analytics engineers (Highly recommended).
• r/dataengineering - Subreddit of data engineering topics.
• Drill to Detail Podcast - Special guests discussing big data, business intelligence, modern data stack.

Sample Projects

Sample projects which work out-of-the box. Reflect use-cases publicly available.

• dag-stack - Dbt-Airflow-GreatExpectations Stack.
• Jaffle Shop - A self-contained dbt project for testing purposes.
• Spotify User Analytics - Sample dbt project with Spotify user data.
• dbt-github-workflow - Deploy BigQuery + Airflow.
• airflow-dbt-demo - Demonstration of Airflow integration.

Contributors

Thanks for all the great resources! Can't see your avatar? Check the contribution guide on how you can submit your resources to the community!