Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Copy-pastable dbt Cloud CI check commands #4663

Merged
merged 5 commits into from
Dec 18, 2023
Merged

Copy-pastable dbt Cloud CI check commands #4663

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
06eab88
Fix spelling
dbeatty10 Dec 15, 2023
9bc167c
Copy-pasteable commands
dbeatty10 Dec 15, 2023
8d04ade
Remove rogue backtick
dbeatty10 Dec 16, 2023
4ca777e
Merge branch 'current' into dbeatty/copy-pasteable-commands
mirnawong1 Dec 18, 2023
7619328
Merge branch 'current' into dbeatty/copy-pasteable-commands
matthewshaver Dec 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions website/docs/best-practices/clone-incremental-models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,17 @@ This can be suboptimal because:
- Typically incremental models are your largest datasets, so they take a long time to build in their entirety which can slow down development time and incur high warehouse costs.
- There are situations where a `full-refresh` of the incremental model passes successfully in your CI job but an _incremental_ build of that same table in prod would fail when the PR is merged into main (think schema drift where [on_schema_change](/docs/build/incremental-models#what-if-the-columns-of-my-incremental-model-change) config is set to `fail`)

You can alleviate these problems by zero copy cloning the relevant, pre-exisitng incremental models into your PR-specific schema as the first step of the CI job using the `dbt clone` command. This way, the incremental models already exist in the PR-specific schema when you first execute the command `dbt build --select state:modified+` so the `is_incremental` flag will be `true`.
You can alleviate these problems by zero copy cloning the relevant, pre-existing incremental models into your PR-specific schema as the first step of the CI job using the `dbt clone` command. This way, the incremental models already exist in the PR-specific schema when you first execute the command `dbt build --select state:modified+` so the `is_incremental` flag will be `true`.

You'll have two commands for your dbt Cloud CI check to execute:
1. Clone all of the pre-existing incremental models that have been modified or are downstream of another model that has been modified: `dbt clone --select state:modified+,config.materialized:incremental,state:old`
2. Build all of the models that have been modified and their downstream dependencies: `dbt build --select state:modified+`
1. Clone all of the pre-existing incremental models that have been modified or are downstream of another model that has been modified:
```shell
dbt clone --select state:modified+,config.materialized:incremental,state:old
```
2. Build all of the models that have been modified and their downstream dependencies:
```shell
dbt build --select state:modified+
```

Because of your first clone step, the incremental models selected in your `dbt build` on the second step will run in incremental mode.

Expand Down
Loading