☁️ Azure Data Factory Customer Managed Key Terraform Module

Encryption configured on an existing factory — where terraform destroy succeeds and changes nothing. Targets hashicorp/azurerm ~> 4.0.

• 🔐 Configures customer-managed-key encryption on an existing Data Factory. It creates nothing: its Resource ID is the factory's.
• 🔴 terraform destroy makes no API call. The provider logs that the key cannot be removed once added and returns — the resource leaves state and the factory stays encrypted.
• 📌 The key identifier must be versioned, so key rotation is a Terraform change rather than something the vault does on its own.
• 👤 Omitting the identity selects the factory's system-assigned identity — a choice, not an absence.
• ⚖️ The factory resource can set the same encryption inline, and this suite's factory module exposes both arguments — prefer that where you own the factory. Configuring both fails the apply; it does not drift quietly.
• 🗝️ Whichever identity is used needs Get, Wrap Key and Unwrap Key on the vault — or, on the RBAC route, two roles, not one — granted elsewhere and unverifiable here.
• 🚫 Three vendor prerequisites the provider cannot see: the factory must be empty, it must not be managed-VNet enabled, and the vault must share its region and Entra tenant.

💡 Why it matters: two of this resource's behaviours are invisible in a plan and irreversible in practice. Enabling customer-managed encryption cannot be undone without recreating the factory, and disabling it in Terraform does not disable it in Azure.

If this module saves you time:

• ⭐ Star the repository — it helps others find it.
• 💼 Connect on LinkedIn — linkedin.com/in/microsoftexpert
• ☕ Buy me a coffee — buymeacoffee.com/microsoftexpert

flowchart TB
  ADF["azurerm_data_factory<br/>ITS OWN ARGUMENTS CAN SET<br/>THE SAME KEY - prefer those"]
  THIS["THIS MODULE<br/>customer managed key<br/>ITS id IS THE FACTORY'S id"]
  KEY["azurerm_key_vault_key<br/>VERSIONED identifier required"]
  KV["azurerm_key_vault<br/>purge protection and soft delete<br/>are the real controls"]
  UAI["user assigned identity<br/>OPTIONAL - omitting selects<br/>the system assigned one"]
  PERM["Get, Wrap Key, Unwrap Key<br/>granted elsewhere, unverifiable here"]

  ADF -->|"data_factory_id"| THIS
  KEY -->|"customer_managed_key_id"| THIS
  KV -->|"holds"| KEY
  UAI -.->|"user_assigned_identity_id"| THIS
  UAI -->|"must ALREADY be assigned to"| ADF
  PERM -.->|"held by the factory's identity"| KV

  classDef me fill:#0078D4,stroke:#004578,color:#ffffff
  classDef keystone fill:#004578,stroke:#002B44,color:#ffffff
  classDef other fill:#F0F3F7,stroke:#B8C4D0,color:#1B1B1B
  class THIS me
  class ADF keystone
  class KEY,KV,UAI,PERM other

Note the top node: the factory can configure this itself — and using both is not a quiet double-write. This resource's create reads the factory first and returns an already exists error whenever the factory carries any encryption configuration, whoever set it, so the second writer fails loudly. The two also disagree on the key form: the factory accepts a versioned or a versionless identifier, while this resource accepts only a versioned one.

flowchart TB
  IN1["var.data_factory_id<br/>the ONLY force-new field"]
  IN2["var.customer_managed_key_id<br/>MUST be VERSIONED"]
  IN3["var.user_assigned_identity_id<br/>omit to use system assigned"]
  R["azurerm_data_factory_customer_managed_key.this<br/>id IS the factory's id"]
  O1["key_vault_name / key_name / key_version"]
  O2["key_rotation_requires<br/>_a_terraform_change"]
  O3["destroy_removes_it_from_state<br/>_but_not_from_azure"]
  O4["uses_user_assigned_identity"]
  O5["the_factory_resource_can_also<br/>_set_a_customer_managed_key"]

  IN1 --> R
  IN2 --> R
  IN3 --> R
  IN2 --> O1
  IN2 --> O2
  R --> O3
  IN3 --> O4
  R --> O5

  classDef me fill:#0078D4,stroke:#004578,color:#ffffff
  classDef io fill:#F0F3F7,stroke:#B8C4D0,color:#1B1B1B
  class R me
  class IN1,IN2,IN3,O1,O2,O3,O4,O5 io

Resource inventory

Schema notes that bite

• 🔴 Delete() makes no API call. It logs that the key cannot be removed from a Data Factory once added and returns. Destroy succeeds, state is cleared, and the factory remains encrypted.
• 🔴 The resource's id is the factory's id. It configures rather than creates, and imports by the factory's ID.
• 🔴 The read hard-errors on a factory with no encryption, where a missing factory is marked gone and dropped from state. So if the factory's encryption is removed out of band — including by the documented remedy of recreating the factory — this resource becomes unrefreshable and plan, apply and destroy all fail until it is removed from state by hand.
• 🔴 An existing encryption configuration fails the create. Before writing, the create fetches the factory and returns an already exists error if any encryption is already configured. The only silent-overwrite path is a caller who has enabled the provider feature that skips import checks, which nothing here can see.
• 🔴 customer_managed_key_id must be VERSIONED on THIS resource — a versionless identifier is rejected, and so is a secret or certificate identifier. This is not a family rule: azurerm_data_factory parses a field of the same name as either form, so a value that is legal inline on the factory is refused here.
• 🔴 The factory resource can set the same encryption inline via customer_managed_key_id and customer_managed_key_identity_id, and the two disagree on identity: this suite's factory module requires a user-assigned identity alongside the key; this resource treats it as optional.
• ⚠️ Omitting user_assigned_identity_id selects the system-assigned identity, not "no identity".
• ⚠️ Only data_factory_id is force-new. The key and identity update in place.
• ⚠️ No name, location or tags.
• ⚠️ No CustomizeDiff, no ConflictsWith, no lock on the factory. The create and update perform a read-modify-write of the factory's whole payload, so a concurrent write from elsewhere can be lost.
• ⚠️ The apply does not wait for the key to be wrapped. One ARM write, no poller — a green apply means the request was accepted, not that encryption is in force.

🔒 The permission that makes this work is not held by whoever runs Terraform. The factory's identity — user-assigned if named here, system-assigned otherwise — needs key permissions on the vault. Nothing here grants or checks them.

🔴 And the RBAC route needs TWO roles, not one. Microsoft's access-policy route grants Get, Unwrap Key and Wrap Key; its role-based route requires Key Vault Crypto Service Encryption User and Key Vault Secrets User. The crypto role on its own is the common mistake, and its failure mode is a factory that cannot wrap its key with no clear cause.

⚠️ Nothing reports which identity actually wraps the key. The resource exposes no effective-identity attribute, so when no user-assigned identity is supplied the principal to grant is the factory's own managed service identity — readable from the factory, not from here. And note there is no delete permission in this table: the provider's delete makes no call, so none is exercised.

• An existing Data Factory whose encryption is not already configured — by the factory resource inline, by the portal, or by a previous apply. If it is, the create fails with already exists; adopt it with import.
• 🔴 An EMPTY Data Factory. Microsoft documents that a customer-managed key can only be configured on a factory containing no linked services, no pipelines and no data flows, and that deploying one to a non-empty factory results in an error. Enable the key immediately after the factory is created.
• 🔴 A factory that is NOT managed-virtual-network enabled. Microsoft states this approach does not work with such factories — and this suite's factory module defaults managed_virtual_network_enabled to true, so a factory built on its defaults is not a candidate. Disabling the flag afterwards is force-new in that direction and destroys the factory, so decide it at creation time.
• The vault in the factory's region and Entra tenant. Microsoft makes region and tenant hard requirements and says explicitly that the two may be in different subscriptions.
• An RSA or RSA-HSM key. No other key type is supported; an EC key is a well-formed identifier that fails at the ARM write.
• A Key Vault key — not a secret, not a certificate — and its versioned identifier.
• Purge protection and soft delete on that vault. Azure requires them for customer-managed keys, and they stand between a deleted key and an unrecoverable factory.
• If a user-assigned identity is named, it must already be assigned to the factory and hold Get, Wrap Key and Unwrap Key (or both RBAC roles above). Omitting it selects the system-assigned identity, which needs the same.

terraform-azurerm-data-factory-customer-managed-key/
├── providers.tf    # required_version + the pinned azurerm; no provider block
├── variables.tf    # four inputs, six validations
├── main.tf         # locals splitting the key URL and reading the identity choice
├── outputs.tf      # 47 outputs, led by the destroy that does nothing
├── README.md       # this file
├── SCOPE.md        # the cross-module contract
├── LICENSE         # MIT
└── .gitignore

provider "azurerm" {
  features {}
}

module "factory_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-customer-managed-key.git?ref=v1.0.0"

  data_factory_id         = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-data-eastus/providers/Microsoft.DataFactory/factories/adf-corp"
  customer_managed_key_id = "https://kv-corp.vault.azure.net/keys/adf-cmk/1a2b3c4d5e6f7890abcdef1234567890"
}

⚠️ Read the destroy behaviour before applying this. Enabling customer-managed encryption cannot be undone without recreating the factory.

Consumes

Emits

module "factory_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-customer-managed-key.git?ref=v1.0.0"

  data_factory_id         = var.data_factory_id
  customer_managed_key_id = var.versioned_key_id
}

output "irreversibility" {
  value = {
    destroy_is_a_no_op   = module.factory_cmk.destroy_removes_it_from_state_but_not_from_azure
    removal_needs_rebuild = module.factory_cmk.removing_the_key_requires_recreating_the_factory
    block_removal_silent = module.factory_cmk.a_removed_module_block_leaves_encryption_in_place
  }
}

module "factory_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-customer-managed-key.git?ref=v1.0.0"

  data_factory_id           = module.data_factory.id
  customer_managed_key_id   = var.versioned_key_id
  user_assigned_identity_id = module.identity.id

  # Redundant, and kept only as documentation of intent -- see the note below.
  depends_on = [module.data_factory]
}

# REJECTED offline:
#   customer_managed_key_id = "https://kv-corp.vault.azure.net/keys/adf-cmk"
#
# So is a SECRET identifier, and a CERTIFICATE identifier.

output "why" {
  value = module.factory_cmk.the_key_identifier_must_be_versioned
}

variable "adf_key_version" {
  type        = string
  description = "Bump this when the key is rotated in the vault, then apply."
}

module "factory_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-customer-managed-key.git?ref=v1.0.0"

  data_factory_id         = var.data_factory_id
  customer_managed_key_id = "https://kv-corp.vault.azure.net/keys/adf-cmk/${var.adf_key_version}"
}

output "rotation_model" {
  value = module.factory_cmk.key_rotation_requires_a_terraform_change
}

variable "expected_key_versionless_id" {
  type        = string
  description = "The key this factory is meant to use, without a version."
}

check "using_the_expected_key" {
  assert {
    condition     = module.factory_cmk.key_versionless_id == var.expected_key_versionless_id
    error_message = "The factory is encrypted with a different key than expected."
  }
}

output "key_detail" {
  value = {
    vault   = module.factory_cmk.key_vault_name
    key     = module.factory_cmk.key_name
    version = module.factory_cmk.key_version
  }
}

output "identity_in_use" {
  value = {
    user_assigned = module.factory_cmk.uses_user_assigned_identity # false => SYSTEM-assigned
    name          = module.factory_cmk.user_assigned_identity_name # null when system-assigned
  }
}

# The FACTORY module can set the same encryption inline:
#
#   module "data_factory" {
#     ...
#     customer_managed_key_id          = var.versioned_key_id
#     customer_managed_key_identity_id = module.identity.id
#   }
#
# Doing that AND using this module FAILS THE APPLY: whichever runs second
# finds the factory already encrypted and errors with `already exists`.

output "check_this_first" {
  value = module.factory_cmk.the_factory_resource_can_also_set_a_customer_managed_key
}

output "unverifiable" {
  value = {
    key_and_permissions = module.factory_cmk.the_module_cannot_verify_the_key_or_the_permissions
    assignment          = module.factory_cmk.the_identity_must_already_be_assigned_to_the_factory
    vault_permissions   = module.factory_cmk.the_identity_needs_key_permissions_on_the_vault
  }
}

output "availability_risk" {
  value = module.factory_cmk.losing_access_to_the_key_disables_the_factory
}

output "singleton" {
  value = {
    one_per_factory = module.factory_cmk.this_is_a_per_factory_singleton
    id_is_factorys  = module.factory_cmk.this_resource_configures_the_factory_rather_than_creating_anything
  }
}

output "change_surface" {
  value = {
    force_new  = module.factory_cmk.force_new_fields                      # ["data_factory_id"]
    updatable  = module.factory_cmk.fields_that_can_change_after_creation # key + identity
    from_azure = module.factory_cmk.fields_azure_returns_on_read
  }
}

import {
  to = module.factory_cmk.azurerm_data_factory_customer_managed_key.this
  id = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-data-eastus/providers/Microsoft.DataFactory/factories/adf-corp"
}

provider "azurerm" {
  features {}
}

# 1. The identity that will reach the key.
module "identity" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-user-assigned-identity.git?ref=v1.0.0"

  name                = "uai-adf-cmk"
  resource_group_name = "rg-data-eastus"
  location            = "eastus"
}

# 2. The factory, with that identity assigned. Note it does NOT set the key
#    inline here -- doing both fails the apply, because this resource refuses
#    to create against a factory that already carries encryption.
module "data_factory" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory.git?ref=v1.0.0"

  name                = "adf-corp"
  resource_group_name = "rg-data-eastus"
  location            = "eastus"

  identity = {
    type         = "UserAssigned"
    identity_ids = [module.identity.id]
  }

  # REQUIRED FOR THIS COMPOSITION TO APPLY. That module defaults this to true
  # as an anti-exfiltration posture, and Microsoft states the customer-managed
  # key approach does not work with a managed-VNet factory. Turning it off
  # later destroys and recreates the factory, so it is decided here.
  managed_virtual_network_enabled = false

  tags = { owner = "data-platform" }
}

# 3. THIS MODULE -- encryption, configured on the factory above.
module "factory_cmk" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-customer-managed-key.git?ref=v1.0.0"

  data_factory_id           = module.data_factory.id
  customer_managed_key_id   = var.versioned_key_id
  user_assigned_identity_id = module.identity.id

  # Redundant: referencing module.data_factory.id already orders this after the
  # factory resource, which is where the identity block lives. Kept as intent.
  depends_on = [module.data_factory]
}

output "encryption_status" {
  description = "What is configured, and what Terraform cannot confirm or undo."
  value = {
    factory          = module.factory_cmk.data_factory_id
    vault            = module.factory_cmk.key_vault_name
    key              = module.factory_cmk.key_name
    version          = module.factory_cmk.key_version
    identity         = module.factory_cmk.user_assigned_identity_name
    reversible       = false # Removing the key requires recreating the factory.
    permissions_ok   = false # Key permissions are granted and checked elsewhere.
  }
}

variable "data_factory_id" {
  type = string
  # /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.DataFactory
  #   /factories/<factory>
  # Anchored at both ends. THIS RESOURCE'S OWN ID IS THIS VALUE.
}

variable "customer_managed_key_id" {
  type = string
  # https://<vault>.vault.azure.net/keys/<key>/<version>
  # Three checks, mirroring the three stages of the provider's own parse:
  #   2-or-3 path segments (a trailing slash is trimmed, so it is allowed);
  #   the type segment must be exactly lower-case `keys`;
  #   the version must be present -- ON THIS RESOURCE ONLY, not on the factory.
  # NOT force-new -- which is what makes rotation possible.
}

variable "user_assigned_identity_id" {
  type    = string
  default = null
  # /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.ManagedIdentity
  #   /userAssignedIdentities/<name>
  # OMITTING IT SELECTS THE FACTORY'S SYSTEM-ASSIGNED IDENTITY -- not "none".
}

variable "timeouts" {
  type = object({
    create = optional(string)
    read   = optional(string)
    update = optional(string)
    delete = optional(string)
  })
  default = null
  # Provider defaults: create 30m, update 30m, read 5m, delete 30m -- though
  # the delete makes no API call, so its timeout bounds nothing. Each value is
  # a Go duration: `1h30m`, `1.5h`, `500ms` and a bare `0` are all accepted.
}

No output is sensitive.

It configures rather than creates, and its Resource ID proves it. This resource has no name, no location and no identity of its own: its id is the Data Factory's, and it is imported by that ID. That makes it a per-factory singleton — and that singleton is enforced, not merely conventional: a second instance against one factory fails its create with an already exists error, because the provider reads the factory first and refuses to write over encryption that is already there.

And the read is where this resource is genuinely dangerous. The provider distinguishes two failures and treats them oppositely: a factory that has been deleted is marked gone and quietly dropped from state, while a factory that exists with no encryption configuration returns a hard error. There is no way back through Terraform once that happens — plan, apply and destroy all refresh first, so all three fail, and only terraform state rm clears it. The state that triggers it is reachable by following the documented remedy for removing the key, which is to delete and recreate the factory: do that while this resource is still in state and the state is stuck.

The destroy is inert, in two ways that are easy to miss. The provider's delete function logs that the key cannot be removed once added and returns without calling Azure. So terraform destroy succeeds, the resource leaves state, and the factory stays encrypted — and separately, simply deleting the module block from a configuration produces a clean plan while leaving Terraform with no record of encryption still in force. That second case is a drift only an import can reconcile. Both are emitted as constants because neither appears anywhere in a plan.

Rotation is a Terraform change, which is the inverse of the credential modules in this family. The provider requires a versioned key identifier, so the factory is pinned to one version. Rotating the key in the vault does not update the factory; the new version must be written here and applied. Get that wrong and nothing warns you — there is no diff, no error, and no drift, until the old version is disabled and the factory loses access to its own metadata. key_versionless_id exists so a check block can confirm the right key is in use without failing on a legitimate rotation.

Omitting the identity selects one rather than none. With no user_assigned_identity_id, the factory's system-assigned identity is used — and needs exactly the same Get, Wrap Key and Unwrap Key permissions on the vault. uses_user_assigned_identity is emitted precisely because false reads as "no identity" and means something else.

And there is a decision to make before adopting this module at all. The Data Factory resource carries customer_managed_key_id and customer_managed_key_identity_id inline, and this suite's factory module exposes both — with a check requiring the identity whenever the key is set, which this resource does not (and which the provider applies at creation only). Configuring both does not drift: it fails. This resource's create fetches the factory and returns an already exists error whenever encryption is already configured, whoever set it. The two also disagree on the key form — the factory takes a versioned or a versionless identifier, this resource only a versioned one. Prefer the factory's own arguments where you own the factory, on the strength of the identity rule and not of removal behaviour, which is equally silent on both paths because the factory's fields are Optional and Computed.

Then there is the case this module was positioned for, which the service refuses. The obvious reason to reach for a separate resource is a factory somebody else owns — and a factory somebody else owns has pipelines in it, which is precisely what Microsoft says will not work: a customer-managed key can only be configured on an empty factory. Nor can the factory be managed-VNet enabled. So the honest scope is narrow: a factory created but not yet populated, whose lifecycle sits in a different configuration from its encryption.

terraform init -backend=false
terraform validate
terraform fmt -check

Pin the source at ?ref=v1.0.0, never a branch. This module is plan-only; a human applies from CI.

⚠️ validate run inside this directory proves the configuration parses and type-checks. It evaluates no variables, so it fires none of the six validation {} blocks — see 🧪 Testing for what does.

Which command fires which check is not interchangeable, and getting it backwards is the usual mistake. There are six validation {} blocks — the anchored factory ID; three on the key identifier (segment count, lower-case keys type, version present); the user-assigned identity ID; and the Go-duration form of each timeouts value. All six read a single variable each, and that is what decides where they fire:

So the Runbook's validate proves the module parses; console against a root module that declares these variables is the harness that actually exercises the checks. Terraform also skips a validation whose referenced variable has already failed, so a short error list is not evidence a check is missing.

Every widening needs its own accepting case, or the check quietly narrows over time. Three values the provider accepts are accepted here deliberately and should stay proven: a key identifier with a trailing slash (the provider's parser trims one before counting segments), a dotless vault host (no .vault.azure.net assertion exists anywhere in the parser), and a Managed HSM URL (whose support is unconfirmed either way, so it is not refused). Likewise 1h30m, 1.5h, 500ms and a bare 0 are all legal Go durations and are all accepted — a ^[0-9]+(s|m|h)$ shape would refuse every one of them, and a failed validation blocks destroy.

The key identifier deserves four negatives rather than one, because every plausible wrong value is well-formed: a versionless key ID, a secret ID, a certificate ID, and an ARM Resource ID. Each must be refused. And every value derived from the identity needs its null branch asserted, since the common case is that no user-assigned identity is supplied at all.

What no offline check can reach: whether the factory exists, whether it is empty, whether it is managed-VNet enabled, whether the vault shares its region and tenant, whether the key exists and is RSA, whether the identity is assigned to the factory, and whether it holds the vault permissions. Nor can any test observe the inert delete, the already exists guard on create, or the read's hard error on missing encryption — all three are read from the provider's source rather than exercised.

id                      = "/subscriptions/00000000-.../providers/Microsoft.DataFactory/factories/adf-corp"
data_factory_id         = "/subscriptions/00000000-.../providers/Microsoft.DataFactory/factories/adf-corp"
data_factory_name       = "adf-corp"
customer_managed_key_id = "https://kv-corp.vault.azure.net/keys/adf-cmk/1a2b3c4d5e6f7890abcdef1234567890"
key_vault_name          = "kv-corp"
key_name                = "adf-cmk"
key_version             = "1a2b3c4d5e6f7890abcdef1234567890"
key_versionless_id      = "https://kv-corp.vault.azure.net/keys/adf-cmk"
uses_user_assigned_identity = true
user_assigned_identity_name = "uai-adf-cmk"
force_new_fields        = ["data_factory_id"]
fields_that_can_change_after_creation = ["customer_managed_key_id", "user_assigned_identity_id"]
destroy_removes_it_from_state_but_not_from_azure = true

• azurerm_data_factory_customer_managed_key
• Encrypt Azure Data Factory with customer-managed keys
• Azure Key Vault soft delete and purge protection
• Sibling modules: terraform-azurerm-data-factory, terraform-azurerm-key-vault, terraform-azurerm-user-assigned-identity
• This module's SCOPE.md

💙 "Infrastructure as Code should be standardized, consistent, and secure."