Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project flags, for global config defaults + behavior change migrations #4814

Merged
merged 42 commits into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
c754294
First draft: global configs --> flags
jtcohen6 Jan 28, 2024
5a66d8c
Fix broken links
jtcohen6 Feb 19, 2024
0ae80fa
PR feedback
jtcohen6 Feb 21, 2024
5953e66
addinb new indirect selection pages
runleonarun Feb 22, 2024
274a7e0
Merge branch 'current' into indirect-selection4773
runleonarun Feb 22, 2024
2138588
adding to sidebar
runleonarun Feb 22, 2024
27198cc
closing file
runleonarun Feb 22, 2024
9ca6e38
Merge branch 'indirect-selection4773' of github.com:dbt-labs/docs.get…
runleonarun Feb 22, 2024
874d911
fixing the env var example
runleonarun Feb 22, 2024
2fc4a11
Merge branch 'current' into indirect-selection4773
runleonarun Feb 22, 2024
86b8320
fix example sentences
runleonarun Feb 22, 2024
8d05854
Update website/dbt-versions.js
runleonarun Feb 22, 2024
923bbbf
Apply suggestions from code review
runleonarun Feb 23, 2024
0115fbb
Apply suggestions from code review
runleonarun Feb 23, 2024
0a63a33
Update website/docs/reference/global-configs/indirect-selection.md
runleonarun Feb 23, 2024
6711079
Apply suggestions from code review
dbeatty10 Feb 23, 2024
7d14a93
Use relative links instead of absolute
dbeatty10 Feb 23, 2024
9dafc52
Apply suggestions from code review
matthewshaver Feb 26, 2024
9b2fceb
Update website/docs/reference/global-configs/about-global-configs.md
matthewshaver Feb 26, 2024
f496f90
Merge branch 'jerco/flags-behavior-changes' into indirect-selection4773
runleonarun Feb 26, 2024
292b20e
Merge branch 'current' into jerco/flags-behavior-changes
runleonarun Feb 26, 2024
4259f83
Merge branch 'jerco/flags-behavior-changes' into indirect-selection4773
runleonarun Feb 26, 2024
b434f2f
Update profiles.yml.md
runleonarun Feb 26, 2024
a53d79a
Apply suggestions from code review
runleonarun Feb 26, 2024
02f2113
adding backindirect selection
runleonarun Feb 26, 2024
b29c9d5
Update website/sidebars.js to include record-timing-info
dbeatty10 Feb 26, 2024
72a379a
Apply suggestions from code review
runleonarun Feb 26, 2024
3bcf36b
Updating profiles.yml
matthewshaver Feb 26, 2024
e38d6ef
Merge branch 'indirect-selection4773' into profiles-global-configs
matthewshaver Feb 26, 2024
d1c8216
Merge branch 'jerco/flags-behavior-changes' into indirect-selection4773
matthewshaver Feb 26, 2024
28c6049
Merge branch 'indirect-selection4773' into profiles-global-configs
matthewshaver Feb 26, 2024
39dd7fc
Merge branch 'current' into jerco/flags-behavior-changes
matthewshaver Feb 26, 2024
3970c51
Merge branch 'jerco/flags-behavior-changes' into indirect-selection4773
matthewshaver Feb 26, 2024
54b3d5a
Merge branch 'indirect-selection4773' into profiles-global-configs
matthewshaver Feb 26, 2024
299bd00
Profiles global configs (#4992)
matthewshaver Feb 26, 2024
1faefd2
Adding new indirect selection pages (#4977)
matthewshaver Feb 27, 2024
6f2543c
Update sidebars.js
matthewshaver Feb 27, 2024
0c5a3f1
Update about-global-configs.md
matthewshaver Feb 27, 2024
6fa4e89
Merge branch 'current' into jerco/flags-behavior-changes
matthewshaver Feb 27, 2024
4cd558a
Update sidebars.js
matthewshaver Feb 27, 2024
8e4b378
Apply suggestions from code review
matthewshaver Feb 27, 2024
40066f8
Merge branch 'current' into jerco/flags-behavior-changes
matthewshaver Feb 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,5 @@ Uncomment when publishing docs for a prerelease version of dbt:
Adding or removing pages (delete if not applicable):
- [ ] Add/remove page in `website/sidebars.js`
- [ ] Provide a unique filename for new pages
- [ ] Add an entry for deleted pages in `website/static/_redirects`
- [ ] Add an entry for deleted pages in `website/vercel.json`
dbeatty10 marked this conversation as resolved.
Show resolved Hide resolved
- [ ] Run link testing locally with `npm run build` to update the links that point to deleted pages
4 changes: 4 additions & 0 deletions website/dbt-versions.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ exports.versions = [
]

exports.versionedPages = [
{
"page": "reference/global-configs/indirect-selection",
"firstVersion": "1.8",
},
{
"page": "reference/resource-configs/store_failures_as",
"firstVersion": "1.7",
Expand Down
42 changes: 41 additions & 1 deletion website/docs/docs/core/connect-data-platform/profiles.yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,14 @@ If you're using dbt Cloud, you can [connect to your data platform](/docs/cloud/c

This section identifies the parts of your `profiles.yml` that aren't specific to a particular data platform. For specific connection details, refer to the relevant page for your data platform.

<VersionBlock lastVersion="1.7">

:::warning Global configs

Starting in dbt v1.8, global configs have been deprecated from the `profiles.yml` file and should be configured in the [`dbt_project.yml`](/reference/dbt_project.yml) file instead.

:::

<File name='profiles.yml'>

```yml
Expand All @@ -28,6 +36,7 @@ This section identifies the parts of your `profiles.yml` that aren't specific to
[debug](/reference/global-configs/logs#log-level): <true | false>
[version_check](/reference/global-configs/version-compatibility): <true | false>
[fail_fast](/reference/global-configs/failing-fast): <true | false>
[indirect_selection](/reference/global-configs/indirect-selection): <eager | cautious | buildable | empty>
[use_experimental_parser](/reference/global-configs/parsing): <true | false>
[static_parser](/reference/global-configs/parsing): <true | false>
[cache_selected_only](/reference/global-configs/cache): <true | false>
Expand All @@ -54,6 +63,37 @@ This section identifies the parts of your `profiles.yml` that aren't specific to

</File>

</VersionBlock>

<VersionBlock firstVersion="1.8">

<File name='profiles.yml'>

```yml

<profile-name>:
target: <target-name> # this is the default target
outputs:
<target-name>:
type: <bigquery | postgres | redshift | snowflake | other>
schema: <schema_identifier>
threads: <natural_number>

### database-specific connection details
...

<target-name>: # additional targets
...

<profile-name>: # additional profiles
...

```

</File>

</VersionBlock>

## User config

You can set default values of global configs for all projects that you run using your local machine. See the docs on [global configs](/reference/global-configs/about-global-configs) for details.
You can set default values of global configs for all projects that you run using your local machine. Refer to [About global configs](/reference/global-configs/about-global-configs) for details.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion website/docs/docs/dbt-cloud-apis/sl-manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ dbt creates an [artifact](/reference/artifacts/dbt-artifacts) file called the _S

By using the semantic manifest produced by dbt Core, MetricFlow will instantiate a data flow plan and generate SQL from Semantic Layer query requests. It's a valuable reference that you can use to understand the structure and details of your data models.

Similar to the [`manifest.json` file](/reference/artifacts/manifest-json), the `semantic_manifest.json` file also lives in the [target directory](/reference/project-configs/target-path) of your dbt project where dbt stores various artifacts (such as compiled models and tests) generated during the execution of your project.
Similar to the [`manifest.json` file](/reference/artifacts/manifest-json), the `semantic_manifest.json` file also lives in the [target directory](/reference/global-configs/json-artifacts) of your dbt project where dbt stores various artifacts (such as compiled models and tests) generated during the execution of your project.

## How it's produced

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ See GitHub discussion [dbt-labs/dbt-core#5468](https://github.com/dbt-labs/dbt-c
- **[Metrics](/docs/build/build-metrics-intro)** now support an `expression` type (metrics-on-metrics), as well as a `metric()` function to use when referencing metrics from within models, macros, or `expression`-type metrics. For more information on how to use expression metrics, check out the [**`dbt_metrics` package**](https://github.com/dbt-labs/dbt_metrics)
- **[dbt-Jinja functions](/reference/dbt-jinja-functions)** now include the [`itertools` Python module](/reference/dbt-jinja-functions/modules#itertools), as well as the [set](/reference/dbt-jinja-functions/set) and [zip](/reference/dbt-jinja-functions/zip) functions.
- **[Node selection](/reference/node-selection/syntax)** includes a [file selection method](/reference/node-selection/methods#the-file-method) (`-s model.sql`), and [yaml selector](/reference/node-selection/yaml-selectors) inheritance.
- **[Global configs](/reference/global-configs/about-global-configs)** now include CLI flag and environment variable settings for [`target-path`](/reference/project-configs/target-path) and [`log-path`](/reference/project-configs/log-path), which can be used to override the values set in `dbt_project.yml`
- **[Global configs](/reference/global-configs/about-global-configs)** now include CLI flag and environment variable settings for [`target-path`](/reference/global-configs/json-artifacts) and [`log-path`](/reference/global-configs/logs), which can be used to override the values set in `dbt_project.yml`

### Specific adapters

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ Several under-the-hood changes from past minor versions, tagged with deprecation
- Add [metrics](/docs/build/build-metrics-intro), a new node type
- [Generic tests](/best-practices/writing-custom-generic-tests) can be defined in `tests/generic` (new), in addition to `macros/` (as before)
- [Parsing](/reference/parsing): partial parsing and static parsing have been turned on by default.
- [Global configs](/reference/global-configs/about-global-configs) have been standardized. Related updates to [global CLI flags](/reference/global-cli-flags) and [`profiles.yml`](/docs/core/connect-data-platform/profiles.yml).
- [Global configs](/reference/global-configs/about-global-configs) have been standardized. Related updates to [global CLI flags](/reference/global-configs/about-global-configs) and [`profiles.yml`](/docs/core/connect-data-platform/profiles.yml).
- [The `init` command](/reference/commands/init) has a whole new look and feel. It's no longer just for first-time users.
- Add `result:<status>` subselectors for smarter reruns when dbt models have errors and tests fail. See examples: [Pro-tips for Workflows](/best-practices/best-practice-workflows#pro-tips-for-workflows)
- Secret-prefixed [env vars](/reference/dbt-jinja-functions/env_var) are now allowed only in `profiles.yml` + `packages.yml`
2 changes: 1 addition & 1 deletion website/docs/reference/artifacts/dbt-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Most dbt commands (and corresponding RPC methods) produce artifacts:

## Where are artifacts produced?

By default, artifacts are written to the `/target` directory of your dbt project. You can configure the location using the [`target-path`](/reference/project-configs/target-path#configuration).
By default, artifacts are written to the `/target` directory of your dbt project. You can configure the location using the [`target-path` flag](/reference/global-configs/json-artifacts).

## Common metadata

Expand Down
15 changes: 9 additions & 6 deletions website/docs/reference/dbt_project.yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ The following example is a list of all available configurations in the `dbt_proj
[docs-paths](/reference/project-configs/docs-paths): [directorypath]
[asset-paths](/reference/project-configs/asset-paths): [directorypath]

[target-path](/reference/project-configs/target-path): directorypath
[log-path](/reference/project-configs/log-path): directorypath
[target-path](/reference/global-configs/json-artifacts): directorypath
[log-path](/reference/global-configs/logs): directorypath
[packages-install-path](/reference/project-configs/packages-install-path): directorypath

[clean-targets](/reference/project-configs/clean-targets): [directorypath]
Expand All @@ -52,6 +52,9 @@ The following example is a list of all available configurations in the `dbt_proj

[require-dbt-version](/reference/project-configs/require-dbt-version): version-range | [version-range]

[flags](/reference/global-configs/project-flags):
[<global-configs>](/reference/global-configs/project-flags)

[dbt-cloud](/docs/cloud/cloud-cli-installation):
[project-id](/docs/cloud/configure-cloud-cli#configure-the-dbt-cloud-cli): project_id # Required
[defer-env-id](/docs/cloud/about-cloud-develop-defer#defer-in-dbt-cloud-cli): environment_id # Optional
Expand Down Expand Up @@ -120,8 +123,8 @@ vars:
[docs-paths](/reference/project-configs/docs-paths): [directorypath]
[asset-paths](/reference/project-configs/asset-paths): [directorypath]

[target-path](/reference/project-configs/target-path): directorypath
[log-path](/reference/project-configs/log-path): directorypath
[target-path](/reference/global-configs/json-artifacts): directorypath
[log-path](/reference/global-configs/logs): directorypath
[packages-install-path](/reference/project-configs/packages-install-path): directorypath

[clean-targets](/reference/project-configs/clean-targets): [directorypath]
Expand Down Expand Up @@ -193,8 +196,8 @@ vars:
[docs-paths](/reference/project-configs/docs-paths): [directorypath]
[asset-paths](/reference/project-configs/asset-paths): [directorypath]

[target-path](/reference/project-configs/target-path): directorypath
[log-path](/reference/project-configs/log-path): directorypath
[target-path](/reference/global-configs/json-artifacts): directorypath
[log-path](/reference/global-configs/logs): directorypath
[packages-install-path](/reference/project-configs/packages-install-path): directorypath

[clean-targets](/reference/project-configs/clean-targets): [directorypath]
Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/events-logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: "Events and logs"

As dbt runs, it generates events. The most common way to see those events is as log messages, written in real time to two places:
- The command line terminal (`stdout`), to provide interactive feedback while running dbt.
- The debug log file (`logs/dbt.log`), to enable detailed [debugging of errors](/guides/debug-errors) when they occur. The text-formatted log messages in this file include all `DEBUG`-level events, as well as contextual information, such as log level and thread name. The location of this file can be configured via [the `log_path` config](/reference/project-configs/log-path).
- The debug log file (`logs/dbt.log`), to enable detailed [debugging of errors](/guides/debug-errors) when they occur. The text-formatted log messages in this file include all `DEBUG`-level events, as well as contextual information, such as log level and thread name. The location of this file can be configured via [the `log-path` flag](/reference/global-configs/logs).

<File name='CLI'>

Expand Down
26 changes: 0 additions & 26 deletions website/docs/reference/global-cli-flags.md

This file was deleted.

Loading
Loading