dbt (data build tool)
dbt is the standard for analytics engineering in GTM. It lets you define transformations as modular SQL files, compile them into a DAG, and run them in your warehouse without moving data. Models are version-controlled in Git, tested with built-in assertions, and documented inline. dbt Cloud adds scheduling, CI checks, and a lineage explorer. Teams use it to build customer 360 views, attribution models, and metric layers that feed dashboards and reverse-ETL syncs.
Why it ranks here
- Industry-standard transformation layer; most data teams expect dbt in the stack, and hiring for dbt skills is easier than for proprietary ETL tools.
- Macro library (dbt-utils, dbt-expectations) provides reusable patterns for deduplication, surrogate keys, and data quality tests.
- Version control and CI checks catch breaking changes before they hit production; rollback is a git revert.
Where it sits
- SQLMesh offers faster incremental builds and a richer CLI but has a smaller community and fewer pre-built packages.
- Coalesce provides a visual DAG builder for non-SQL teams but adds a proprietary layer on top of the warehouse.
- Custom SQL scripts in cron jobs work for simple cases but lack testing, lineage, and incremental refresh logic.
How GTM Systems teams use it
- Build staging models that clean and deduplicate raw Salesforce and marketing data, then layer fact and dimension tables on top for BI tools to query.
- Define data quality tests (unique, not_null, relationships) that fail the build if upstream data changes break downstream dashboards.
- Use incremental models to process only new rows since the last run, cutting warehouse costs and transformation time for large event tables.
- Generate documentation sites that show model lineage, column descriptions, and test coverage; share with stakeholders to build trust in the data.
In-depth notes
- dbt Core is open-source and free; dbt Cloud adds scheduling, IDE, and lineage UI with per-developer-seat pricing.
- Models compile to warehouse-native SQL (Snowflake, BigQuery, Redshift, Databricks); dbt itself does not store or move data.
- The ref() macro generates fully-qualified table names and builds the DAG automatically; changing upstream models triggers downstream rebuilds.
- Seeds (small CSV files checked into Git) are useful for mapping tables like territory assignments or product categories.
- dbt Metrics (semantic layer) is now deprecated in favor of MetricFlow; teams building metric layers should evaluate MetricFlow or external tools like Cube.
Best for
Any team building a centralized GTM data model in Snowflake, BigQuery, Redshift, or Databricks; the default choice for analytics engineering.
Avoid if
Your transformations are trivial (one or two tables) or you need real-time streaming; dbt is batch-oriented and adds overhead for simple use cases.
The rest of Data warehouse & modeling
Snowflake Leader Cloud data warehouse with compute-separate-from-storage architecture; dominant choice for GTM analytics and cross-system reporting. #2 
Google BigQuery Leader Serverless cloud data warehouse on GCP; pay-per-query pricing and tight integration with Google Analytics and Sheets. #4 
Databricks Strong Unified lakehouse platform combining data warehousing, streaming, and ML; built on Apache Spark and Delta Lake. #5 
Microsoft Fabric Strong Unified analytics platform combining data warehouse (Synapse), lakehouse, data integration, and Power BI; Azure-native. #7 
ClickHouse Situational Open-source columnar database optimized for real-time analytics; extremely fast for aggregations on high-volume event data. #8 SQLMesh Situational Data transformation framework focused on incremental builds, fast CI, and column-level lineage; positioned as a next-generation dbt alternative. #9 
Coalesce Situational Visual data transformation platform built on Snowflake; provides a drag-and-drop DAG builder that compiles to dbt-compatible SQL. #10 
MotherDuck Situational Serverless cloud service for DuckDB; combines local query execution with cloud-based collaboration and storage. #11 
Firebolt Niche Cloud data warehouse optimized for sub-second query latency on large datasets; uses sparse indexing and aggressive caching. Learning guide
Setup
Install dbt Core via pip or Homebrew, run dbt init to create a project skeleton, and configure profiles.yml with your warehouse credentials (Snowflake account, BigQuery keyfile, or Redshift endpoint). Run dbt debug to verify the connection succeeds.
First thing to build
Create a staging model in models/staging/stg_salesforce_opportunities.sql that selects and renames columns from the raw Opportunities table. Run dbt run to materialize it as a view, then add a uniqueness test on Id and run dbt test to validate.
What actually matters
- Materializations: use views for lightweight transformations and tables for models queried by dashboards; incremental models cut costs on large event tables.
- Tests: add not_null and unique tests to every primary key; use relationships tests to enforce foreign key constraints across models.
- Seeds: store small mapping tables (territory assignments, product SKUs) as CSV files in data/ and reference them with ref('seed_name').
- Documentation: write column descriptions in schema.yml files and run dbt docs generate to produce a lineage site you can share with stakeholders.
Watch out
Running dbt run without a target filter rebuilds every model in the project; use dbt run --select +model_name+ to build only the model and its dependencies.