Skip to content

About

Terraform module: terraform-databricks-metastore-assignment

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

🧱 Databricks Metastore Assignment Terraform Module

Binds an existing Unity Catalog metastore to an existing Databricks workspace at the account plane, against the databricks/databricks provider ~> 1.117.0.

Terraform Provider Module Type Resources Posture

🧩 Overview

  • 🔗 Wires exactly one relationship: a metastore → a workspace.
  • 🚫 No independent identity of its own beyond the relationship (though see the id correction below — the resource does export a real, importable composite ID).
  • 🌐 Account-plane operation — requires an account-level provider context.
  • 🔁 Reassignable — a metastore can be assigned to many workspaces over time; a workspace's assignment can be moved to a different metastore.

💡 Why it matters: every Unity Catalog object a workspace can see — catalogs, schemas, volumes — resolves against whichever metastore is assigned to that workspace. This module is the single point where that binding happens, and it is kept separate from terraform-databricks-metastore because the binding's lifecycle (reassignable, many-to-many over time) does not fit the keystone-owns-its-children shape used elsewhere in this library.


❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!


🗺️ Where this fits

flowchart LR
 META["terraform-databricks-metastore"]
 style META fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 THIS["terraform-databricks-metastore-assignment"]
 style THIS fill:#FF3621,color:#fff,stroke:#1B3139,stroke-width:1px

 WS["Workspace provisioning — external, not part of this catalog"]
 style WS fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px

 CATALOG["terraform-databricks-catalog"]
 style CATALOG fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px

 META -->|"id becomes metastore_id"| THIS
 WS -.->|"workspace_id supplied by caller, not a Terraform reference"| THIS
 THIS -.->|"binds metastore to workspace (no direct reference)"| CATALOG
Loading

terraform-databricks-metastore is this module's keystone/target sibling — its id output feeds this module's metastore_id input directly. workspace_id is genuinely external: workspace provisioning (azurerm_databricks_workspace or the AWS/GCP equivalent) is a different provider's module library entirely and is out of scope for this Databricks catalog. terraform-databricks-catalog depends on this binding existing but has no direct Terraform reference to it — Unity Catalog resolves a catalog against whichever metastore is assigned to the workspace at apply time.

🧬 What this builds

flowchart LR
 subgraph INPUTS["var.* — no keystone, relationship module"]
 MID["metastore_id"]
 WID["workspace_id"]
 end

 REL["databricks_metastore_assignment.this"]
 style REL fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 subgraph OUTPUTS["outputs"]
 ID["id (composite: workspace_id|metastore_id)"]
 OWID["workspace_id (echo)"]
 OMID["metastore_id (echo)"]
 end

 MID --> REL
 WID --> REL
 REL --> ID
 REL --> OWID
 REL --> OMID
Loading

No keystone — relationship module. databricks_metastore_assignment.this is the only resource; it creates no independent object beyond the binding itself. It is still named this (rather than named for a role, the way databricks_grants/databricks_permissions are) because this module has exactly one resource type and exactly one instance per call.

✅ Provider / Versions

Requirement Value
Terraform >= 1.12.0
databricks/databricks ~> 1.117.0
Provider block None — the caller's root module configures provider "databricks" {}
tags / custom_tags Not supported by databricks_metastore_assignment — none added
timeouts Not supported by databricks_metastore_assignment — none added

Schema notes that bite:

  • workspace_id is typed number in the pinned schema, not string. A common copy-paste trap from other resources' string-typed workspace references — get this wrong and terraform validate fails the first time a caller quotes the workspace ID.
  • This resource genuinely exports a meaningful id, of the form <workspace_id>|<metastore_id>, used for terraform import. An earlier draft of this module's SCOPE.md assumed (per this library's general framing of aggregation modules) that no meaningful id existed here — live provider documentation corrected that during authoring. outputs.tf leads with id, following this library's default convention, not the no-id exception.
  • default_catalog_name is present on the resource but provider-deprecated in favor of databricks_default_namespace_setting (a resource not yet in this catalog) — deliberately not exposed as a module variable.
  • The schema also exposes an api ("account" / "workspace") attribute and a provider_config { workspace_id } block for overriding which plane a single resource instance targets. This module deliberately does not expose either as a variable — the caller's account-level provider block is expected to determine plane; a per-resource override here would contradict this library's authentication model.

🔑 Required Databricks Permissions & Scopes

  • Account admin. Metastore assignment is a documented account-level administrative operation across Azure/AWS/GCP Databricks deployments. The live provider documentation for this resource confirms it "can be used with an account or workspace-level provider" but does not itself state an explicit permission model in its Argument/Attribute Reference text — the account-admin requirement is stated from established Databricks platform behavior, not a line quoted directly from this resource's own doc page.

Databricks Prerequisites

  • Account-plane provider context — host = "https://accounts.azuredatabricks.net" (Azure) or "https://accounts.cloud.databricks.com" (AWS/GCP). This module accepts no auth-shaped variables.
  • The referenced metastore (metastore_id) must already exist — created by terraform-databricks-metastore or equivalent.
  • The referenced workspace (workspace_id) must already exist and be enabled for Unity Catalog, provisioned by the caller's cloud-provider-level workspace library (out of scope here).

📁 Module Structure

terraform-databricks-metastore-assignment/
├── providers.tf # required_providers only — no provider {} block
├── variables.tf # metastore_id (string), workspace_id (number)
├── main.tf # databricks_metastore_assignment.this
├── outputs.tf # id first, then workspace_id and metastore_id echoes
├── SCOPE.md # cross-module contract
├── README.md # this file
└── examples/
 └── basic/
 └── main.tf # smallest real, runnable call

⚙️ Quick Start

module "metastore_assignment" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.metastore.id
  workspace_id = 123456789012345
}

ℹ️ This is an account-plane module. The caller's root module must configure a provider with host = "https://accounts.azuredatabricks.net" (Azure) or "https://accounts.cloud.databricks.com" (AWS/GCP) and an account_id — this module accepts neither.

🔌 Cross-Module Contract

Consumes:

Input Type Source module
metastore_id string terraform-databricks-metastore output id
workspace_id number External — caller-supplied; workspace provisioning is not part of this catalog

Emits:

Output Description Consumed by
id Composite assignment ID, <workspace_id>|<metastore_id> — used for terraform import Auditing / drift-detection tooling
workspace_id Echo of the input workspace_id this assignment binds to Downstream root modules confirming which workspace an apply targeted
metastore_id Echo of the input metastore_id this assignment binds Auditing / drift-detection tooling

📚 Example Library

1 · Minimal assignment
module "metastore_assignment" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = "12345678-90ab-cdef-1234-567890abcdef"
  workspace_id = 123456789012345
}
2 · Assigning a metastore to multiple workspaces
module "assign_analytics" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = "12345678-90ab-cdef-1234-567890abcdef"
  workspace_id = 111111111111111
}

module "assign_ml" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = "12345678-90ab-cdef-1234-567890abcdef"
  workspace_id = 222222222222222
}

💡 A single metastore commonly serves many workspaces — this is the intended, supported topology, not an edge case.

3 · Assignment for a regional metastore
module "assign_eu_workspace" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
  workspace_id = 333333333333333
}
4 · Using the metastore module's output directly
module "metastore" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"

  name         = "casey-primary-metastore"
  region       = "eastus"
  storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
}

module "metastore_assignment" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.metastore.id
  workspace_id = 123456789012345
}
5 · Reassigning a workspace to a different metastore (documentation pattern)
module "assign_workspace_to_new_metastore" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.new_metastore.id # change from the old metastore's id
  workspace_id = 123456789012345
}

⚠️ Reassigning a workspace to a different metastore changes which Unity Catalog objects that workspace can see. Review the workspace's active catalog usage before reassigning in a production environment.

6 · Multiple metastore-assignment modules across a fleet of workspaces
locals {
  fleet_workspaces = {
    analytics = 111111111111111
    ml        = 222222222222222
    finance   = 333333333333333
  }
}

module "assignments" {
  for_each = local.fleet_workspaces
  source   = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.metastore.id
  workspace_id = each.value
}

ℹ️ for_each is applied at the caller's root-module level here, not inside this module — this module itself has no child collection to iterate; each instance binds exactly one workspace.

7 · Referencing the assignment's composite id downstream
module "metastore_assignment" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.metastore.id
  workspace_id = 123456789012345
}

output "assignment_import_id" {
  value = module.metastore_assignment.id # "<workspace_id>|<metastore_id>"
}
8 · Minimal least-privilege baseline (recommended starting point)
module "metastore_assignment" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.metastore.id
  workspace_id = var.workspace_id
}

💡 This module has no optional configuration surface — every call is already minimal. Breadth in this module's example library comes from composition (how many workspaces, which metastore), not from configuration variants.

🏗️ 9 · End-to-end composition — metastore → assignment → catalog
module "metastore" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"

  name         = "casey-primary-metastore"
  region       = "eastus"
  storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
  owner        = "uc-admins"
}

module "metastore_assignment" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.metastore.id
  workspace_id = 123456789012345
}

module "analytics_catalog" {
  source = "git::https://github.com/microsoftexpert/terraform-databricks-catalog.git?ref=v1.0.0"

  name = "analytics"

  depends_on = [module.metastore_assignment]
}

ℹ️ terraform-databricks-catalog is a seeded module in this same catalog batch — this composition reflects its planned contract (per its SCOPE.md), not yet a verified cross-module terraform plan. depends_on is required because the catalog module has no direct Terraform reference to the assignment — Unity Catalog resolves the catalog against whichever metastore is assigned to the workspace at apply time, an implicit dependency Terraform's graph cannot see from resource references alone.

📥 Inputs

Variable Type Default Notes
metastore_id string — (required) From terraform-databricks-metastore output id
workspace_id number — (required) Caller-supplied; not a string
Full variable declarations
variable "metastore_id" {
  type = string
  # validation: must not be empty
}

variable "workspace_id" {
  type = number
  # validation: must be > 0
}

🧾 Outputs

Output Description Sensitive?
id Composite assignment ID, <workspace_id>|<metastore_id> No
workspace_id Echo of the workspace_id input No
metastore_id Echo of the metastore_id input No

🧠 Architecture Notes

  • No for_each, no keystone. This module has exactly one resource and no child collection — the entire module is one relationship.
  • id is a genuine, meaningful output, not a fabricated one — it is the resource's own documented composite identifier, used for terraform import. Treat it as a first-class output, not an afterthought.
  • workspace_id must be a number. The most common authoring mistake here is quoting it as a string by habit from other resources' workspace_id-named attributes (e.g. the provider_config block's workspace_id, which is a string in this same schema — a genuinely confusing inconsistency within the provider itself).
  • default_catalog_name, api, and provider_config are intentionally absent from this module's surface — see "Schema notes that bite" above for the reasoning behind each.

🧱 Design Principles

This module has no boolean/enum secure-default surface — its only two inputs are both required identifiers with no permissive/restrictive reading. The secure-by-default discipline here is architectural rather than value-based: the module creates no default catalog binding (default_catalog_name is deliberately unexposed) and no plane-override escape hatch (api / provider_config deliberately unexposed), keeping the account-vs-workspace plane boundary exactly where the caller's provider configuration puts it.

🚀 Runbook

cd terraform-databricks-metastore-assignment
terraform init -backend=false
terraform validate
terraform fmt -check

Pin consumers to an immutable tag — ?ref=v1.0.0 — never a branch. This module is plan-only; a human applies from CI after review, against a sub-production environment first.

🧪 Testing

terraform validate / terraform fmt -check catch: missing metastore_id/workspace_id, the workspace_id > 0 / metastore_id non-empty validations, and a workspace_id passed as a quoted string instead of a number. They do not catch: whether the applying identity actually holds account-admin rights, whether the referenced metastore or workspace actually exist, or whether the workspace is genuinely enabled for Unity Catalog. Those require an actual plan/apply against a live account, out of scope for this authoring process.

💬 Example Output

$ terraform output
id = "123456789012345|12345678-90ab-cdef-1234-567890abcdef"
metastore_id = "12345678-90ab-cdef-1234-567890abcdef"
workspace_id = 123456789012345

🔍 Troubleshooting

Symptom Cause Fix
terraform validate fails with a type-mismatch on workspace_id workspace_id was passed as a quoted string Pass it as a bare number: workspace_id = 123456789012345, not workspace_id = "123456789012345"
Apply fails with a permissions error even though terraform validate passed Applying identity is not an account admin Confirm the identity/service principal running apply holds account-admin rights
Apply fails because the metastore or workspace doesn't exist Referenced metastore_id or workspace_id is wrong or not yet created Confirm terraform-databricks-metastore applied successfully first, and that the workspace ID is correct
Assignment succeeds but the workspace shows no Unity Catalog objects Workspace was not enabled for Unity Catalog by the cloud-provider-level workspace provisioning Confirm Unity Catalog enablement in the workspace's own provisioning module (out of scope here)

🔗 Related Docs

💙 "Infrastructure as Code should be standardized, consistent, and secure."

About

Terraform module: terraform-databricks-metastore-assignment

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages