dbt-core

dbt-core. Is the open-source CLI that transforms data in the warehouse: versioned SQL models, tests, docs, and a DAG executed with dbt build.

Summary

Garden stance: We trial dbt-core for our estate.

When to use: Evaluate on a project when the capability clearly fits the requirement.

When to skip: When a simpler alternative already covers the need.

Details

Project layout

The project is the unit of work. It must include dbt_project.yml (model paths, profiles, vars). Main artifact types:

(Semantic models, metrics, and saved queries apply when using the metrics layer; see current dbt docs for your version.)

CLI reference (common)

• build: run DAG in order (models, tests, seeds, snapshots as configured)
• run: execute models only
• test: run tests (usually after run)
• compile: render SQL without executing
• deps: install package dependencies
• debug: connection and profile diagnostics
• freshness: source freshness checks
• snapshot / seed: run those node types

Incremental models on existing tables

When a table is owned elsewhere but dbt should merge new rows:

1. Model name matches the table (case-sensitive per adapter)
2. materialized='incremental'
3. unique_key set for merge/upsert
4. incremental_strategy (append or merge per adapter)
5. Use is_incremental() and {{ this }} for watermark logic

1
2
3

{% if is_incremental() %}
  AND created_at > (SELECT COALESCE(MAX(scan_timestamp), '1970-01-01'::timestamp) FROM {{ this }})
{% endif %}

See Incremental models.

References

• dbt documentation
• dbt-core on GitHub