Terraform vs DABs: Separation of Concerns
Research notes: 2025-02-25. Boundary tables below still hold. The product name moved: Declarative Automation Bundles (formerly Databricks Asset Bundles / DABs). Docs: https://learn.microsoft.com/azure/databricks/dev-tools/bundles/
Platform opinion distilled from the same Reddit source lives in
lessons-learned/platform-lessons.md. Use this file for the resource tables;
use that file for the why.
Refreshed: 2026-08-08 — the deployment engine and the resource tables are checked against current docs. The boundary rule and the research below still date from the 2025 compile.
What changed since 2025#
Two things move the ground under the tables below. Neither changes my boundary, but both change the reasoning.
- Bundles no longer run on Terraform. Databricks CLI 0.279.0 added a direct deployment
engine that calls the REST API through the Go SDK. New bundles from CLI 1.3.0 use it by
default. Databricks says the Terraform deployment engine "will soon be deprecated". So the
old argument — "bundles are Terraform underneath, so the state guarantees are the same" —
no longer holds. Migrate an existing bundle with
databricks bundle deployment migrate. - Bundles can now deploy Unity Catalog catalogs and external locations. Those two, plus Genie spaces, instance pools, and AI Search endpoints, need the direct engine and do not work on the Terraform engine at all. Catalogs and external locations stay in the Terraform column below by choice, not because bundles cannot do it. The reason is blast radius and repo access: bundles live in the code repo that data engineers can write to.
One gap closed: bundle plan -o json now reports a per-field diff and explains what
triggered each action, and bundle deploy --plan plan.json replays an approved plan. The
"terse failure output vs Terraform" complaint in community-best-practices.md is stale.
Sources, checked 2026-08-08: Migrate to the direct deployment engine.
Sources#
- Alex Ott - Terraform vs. Databricks Asset Bundles
- Databricks Community - Asset bundle vs terraform
- Junaid Effendi - Terraform vs Asset Bundles for Databricks Workflows
- New Math Data - Databricks Asset Bundles Deployment Guide
- Databricks Community - Automate SQL Deployments with Terraform & DABS
- Reddit r/databricks - "What are some things you wish you knew?"
- What are Declarative Automation Bundles? (Azure docs)
The Boundary Rule#
Terraform = "What does the workspace look like?" (infra, identity, catalog structure, security) DABs = "What runs inside the workspace?" (jobs, notebooks, pipelines, compute, secrets)
Keep in Terraform (infrastructure/platform layer)#
| Resource Type | Why Terraform |
|---|---|
| Azure VNet, Subnets, NSGs, NAT | Cloud networking |
| Azure Resource Groups | Cloud infra |
| Azure Storage Account (ADLS Gen2) | Cloud storage |
| Databricks Workspace | Platform provisioning |
| Access Connector | Azure IAM bridge |
| Azure Role Assignments | Cloud IAM |
| Entra ID Groups | Identity provider |
| Databricks Account Groups | Account-level identity |
| Unity Catalog Metastore | Account-level UC |
| Storage Credential | Account-level UC |
| Catalog | Platform-level UC object |
| Schemas + External Locations | Platform-level UC objects |
| Grants (metastore, catalog, schema, external location) | Security/governance — read unity-catalog-grants.md first |
| Volumes (external, managed) | Platform storage |
Catalog grants are the sharp edge here. DABs writes its own grants on catalogs and schemas
when it deploys, so authoritative databricks_grants (plural) wipes them on every apply.
Use databricks_grant (singular) on any object DABs also touches. Full rule in
unity-catalog-grants.md.
Move to DABs (application/project layer)#
| Resource Type | Why DABs |
|---|---|
Jobs (databricks_job) |
Workflow orchestration, tightly coupled to notebook code |
Notebooks (databricks_notebook) |
Code artifacts, should live alongside source code |
Shared Compute Clusters (databricks_cluster) |
Data teams need to iterate on config without TF deploys |
SQL Warehouses (databricks_sql_endpoint) |
Data teams configure sizing/scaling |
Secret Scopes (databricks_secret_scope) |
Application secrets tied to specific pipelines/jobs |
| Cluster Permissions | Tied to cluster lifecycle |
| Job Permissions | Tied to job lifecycle |
| Cluster Policy Permissions | Tied to compute governance |
| Lakeflow Spark Declarative Pipelines (formerly DLT) | Data engineering workflows |
| ML Experiments / Models | Data science workflows |
Key Insights from Research#
- ~~DABs uses Terraform internally — same state management guarantees.~~ Superseded. True when this was written, false now. See "What changed since 2025" above. Judge bundles on the interface and the ownership boundary, not on a shared engine.
- Keeping two repos in sync is painful — practitioners report duplicated work and TF state conflicts when application-level resources live in Terraform.
- Security tradeoff — DAB bundles live in the code repo (visible to developers). For sensitive resources, a separate Terraform repo with restricted access is appropriate. This is why grants and identity stay in Terraform.
- Data team autonomy — data engineers modifying a job schedule or cluster size should not need a Terraform PR. DABs let them own their deployment lifecycle.
- A Databricks Solution Architect recommends using DABs for everything it supports natively, and Terraform only for resources DABs do not support or non-Databricks infrastructure.
- Keeping two repos in sync is painful — practitioners report duplicated work and TF state conflicts when application-level resources live in Terraform.
- Security tradeoff — DABs bundles live in the code repo (visible to developers). For sensitive resources, a separate Terraform repo with restricted access is appropriate. This is why grants and identity stay in Terraform.
- Data team autonomy — data engineers modifying a job schedule or cluster size should not need a Terraform PR. DABs lets them own their deployment lifecycle.
- A Databricks Solution Architect recommends using DABs for everything it supports natively, and Terraform only for resources DABs does not support or non-Databricks infrastructure.