Comments (5)
It is a good idea, we just have a lot on our plate and need to prioritize. I dont know many people running this against 776 tables 😱
We can prioritize with some $ :)
We also help our customers out, so consider Datacoves.com
from dbt-coves.
@BAntonellini
Hey, Bruno. Just wanted to bump here to get thoughts
from dbt-coves.
dbt-osmosis recently implemented something like this and it helped performance immensely.
Fivetran's Salesforce schema alone, for example, has 776 tables. I have a few other large schemas.
Running dbt-osmosis on multiple schemas / DBs becomes extremely slow (hours).
from dbt-coves.
Hey @jaredx435k2d0
We are aware this would be a good addition to dbt-coves, as it would be beneficial for use-cases like yours.
If you feel like contributing, follow our CONTRIBUTING guide and we will review it.
from dbt-coves.
Good to have it acknowledged.
If I had the skills, I'd absolutely do this myself.
I'm learning, so maybe if it's not done in a few months I'll take a stab at it.
from dbt-coves.
Related Issues (20)
- Question: Should `generate sources` remove sources/staging models that are no longer in the database? HOT 2
- [BUG] (question) `generate sources` doesn't pick up new columns HOT 1
- Request:`generate sources`: Quote columns that use SQL keywords (such as `GROUP`) HOT 3
- [feat] Setup task expansion to cover lifecycle of dbt-coves vendored components
- Feature: Generate docs.md files with link to schema.yml columns
- Feature: Get source descriptions from OpenAPI specifications
- [BUG] Arrays are casted as string or record HOT 4
- [BUG] Unable to install v1.3 via PyPI/pip HOT 2
- Feature request: `generate sources` "update" mode also update staging models (rather than overwrite or leave unchanged) HOT 2
- Allow custom model naming HOT 1
- [BUG] dbt-coves fails to detect columns in schema HOT 8
- Option to not generate source.yml by default HOT 1
- Support for dbt 1.5 HOT 1
- [BUG][bigquery] --schema not working for underscore on bigquery
- [BUG] Precommit config installed in `['.']` folder
- Support Pydantic v2.0
- [BUG] Add `sqlalchemy` as a dependency HOT 5
- Make the settings path configurable
- [BUG] Using env_var results in <ContextVar name='DBT_INVOCATION_CONTEXT_VAR' at 0x HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbt-coves.