☁️ Azure Data Factory Dataset — Snowflake Terraform Module

A typed Data Factory dataset describing a table in a Snowflake schema, targeting hashicorp/azurerm ~> 4.0.

• ❄️ Creates one azurerm_data_factory_dataset_snowflake — a named description of a table in a Snowflake schema that pipelines read from and write to.
• 🧬 The Data Factory dataset family's biggest outlier. Its schema_column block is {name, type, precision, scale} — there is no description field — where ten of the twelve siblings use {name, type, description}.
• 🔤 Its column types are 32 Snowflake SQL types that share not one value with the fifteen Data Factory types the siblings use. String is legal there and refused here; VARCHAR is legal here and refused there.
• 🔢 Carries precision and scale, which no other dataset resource has — validated only as non-negative integers, and never checked against each other or the column's type.
• 🗂️ Has a separate schema_name, which the SQL Server Table, MySQL and PostgreSQL datasets lack entirely.

💡 Why it matters: nothing about this resource's column schema can be copied from a sibling in either direction. The harness proves it exhaustively — all 32 Snowflake types and all 15 Data Factory types driven against both this module and a sibling, each accepted by exactly one side.

If this module saved you time:

• ⭐ Star the repository — it is the cheapest signal that this work is worth continuing.
• 💼 Connect on LinkedIn: linkedin.com/in/microsoftexpert
• ☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

flowchart TB
  RG["terraform-azurerm-resource-group"]
  ADF["terraform-azurerm-data-factory"]
  LS["a Snowflake linked service, which chooses the account, warehouse and database"]
  THIS["terraform-azurerm-data-factory-dataset-snowflake"]
  TRIO["the schema-identical trio: sql-server-table, mysql, postgresql"]
  BIN["terraform-azurerm-data-factory-dataset-binary"]
  PIPE["a Data Factory pipeline"]
  TBL["a table in a Snowflake schema"]

  RG -->|"name"| ADF
  ADF -->|"id"| THIS
  ADF -->|"id"| TRIO
  ADF -->|"id"| BIN
  LS -->|"linked_service_name, a bare NAME"| THIS
  THIS -->|"schema_column of name, type, precision, scale"| PIPE
  TRIO -->|"schema_column of name, type, description"| PIPE
  BIN -->|"no schema_column block at all"| PIPE
  THIS -->|"32 SNOWFLAKE SQL types, sharing NOT ONE value"| TRIO
  PIPE -->|"reads only at run time, never here"| TBL

  classDef this fill:#0078D4,stroke:#004578,color:#ffffff,stroke-width:2px
  classDef keystone fill:#004578,stroke:#00243d,color:#ffffff,stroke-width:2px
  classDef sibling fill:#eef3f8,stroke:#b9c8d8,color:#1b2733
  class THIS this
  class ADF keystone
  class RG,LS,TRIO,BIN,PIPE,TBL sibling

Three edges, three different column contracts: this resource's {name, type, precision, scale}, the trio's {name, type, description}, and the Binary dataset's absence of the block altogether.

flowchart TB
  subgraph INPUTS["Inputs"]
    NAME["name (force-new)"]
    ADFID["data_factory_id (force-new)"]
    LSNAME["linked_service_name, a bare NAME"]
    SCH["schema_name, a SEPARATE argument most siblings lack"]
    TBL["table_name, unqualified"]
    COLS["schema_column of name, type, precision, scale, with NO description"]
    META["parameters, additional_properties, annotations, description, folder"]
  end

  THIS["azurerm_data_factory_dataset_snowflake.this"]

  subgraph OUTPUTS["Outputs"]
    OID["id and name"]
    OQT["qualified_table_name, a DISPLAY value only"]
    ODOT["table_name_looks_qualified, the shape a sibling needs and this one does not"]
    OCASE["schema_name_is_lower_case and table_name_is_lower_case"]
    ONUM["numeric_column_definitions_look_wrong, plus its three components"]
    OFACTS["the outlier facts, including the disjoint type vocabulary"]
  end

  NAME --> THIS
  ADFID --> THIS
  LSNAME --> THIS
  SCH --> THIS
  TBL --> THIS
  COLS --> THIS
  META --> THIS

  THIS --> OID
  SCH --> OQT
  TBL --> OQT
  TBL --> ODOT
  SCH --> OCASE
  TBL --> OCASE
  COLS --> ONUM
  THIS --> OFACTS

  classDef this fill:#0078D4,stroke:#004578,color:#ffffff,stroke-width:2px
  classDef sibling fill:#eef3f8,stroke:#b9c8d8,color:#1b2733
  class THIS this
  class NAME,ADFID,LSNAME,SCH,TBL,COLS,META,OID,OQT,ODOT,OCASE,ONUM,OFACTS sibling

Schema notes that bite — verified against the live provider source, not inferred from the schema:

• 🔴 THE COLUMN TYPE SETS ARE DISJOINT. This resource accepts 32 Snowflake SQL types; ten siblings accept 15 Data Factory types; they share not one value. Both are compared case-sensitively.
• 🔴 THE schema_column BLOCK HAS NO description FIELD. A column copied from any sibling is a type error at parse time, not a validation failure — the attribute does not exist on the object.
• 🔴 FOUR TYPE VALUES ARE NOT PLAIN WORDS. DOUBLE PRECISION contains a space; TIMESTAMP_LTZ, TIMESTAMP_NTZ and TIMESTAMP_TZ contain underscores. A list rebuilt by matching letters and digits alone silently loses all four and looks complete.
• 🔴 precision and scale are validated only as integers ≥ 0. They are never compared with each other or with the type, so a scale larger than its precision plans and applies cleanly and is rejected by Snowflake at run time.
• ⚠️ schema_name is a separate argument here. The SQL Server Table, MySQL and PostgreSQL datasets have none — a schema there goes inside table_name. A dotted table_name copied from one of those reaches Snowflake as a single identifier.
• ⚠️ Snowflake folds unquoted identifiers to upper case. public becomes PUBLIC unless the object was created with quotes. This module reports the case rather than normalising it.
• ⚠️ Only name and data_factory_id force replacement.
• ⚠️ Timeouts default to 30m / 5m / 30m / 30m.
• ℹ️ The requires-import error names this resource correctly — the SQL Server Table dataset's names a type that does not exist, and the JSON dataset's names the delimited-text type.
• ℹ️ The registry page runs FLOAT and FLOAT4 together without a separator in its type list. Both are real, distinct values in the provider source.

🔒 No permission on Snowflake is required or used, and none is grantable through Azure RBAC — Snowflake has its own role model entirely outside Azure. Access belongs to the linked service, so whoever can write datasets can describe any table in whatever database and warehouse that linked service points at.

💰 Creating a dataset consumes no Snowflake credits. It starts no warehouse and runs no query. Credits are consumed when a pipeline uses the dataset, which is outside this module.

⚠️ Delete is the operation to review, not create. Removing a dataset breaks every pipeline referencing it by name.

• An existing Data Factory, and its Resource ID.
• A Snowflake linked service already configured in that factory, and its name. It chooses the account, warehouse and database. Nothing here verifies it.
• A schema and table for the dataset to describe — or parameters. Neither is enforced.
• A dataset name meeting Microsoft's Data Factory naming rules. The provider checks only non-emptiness.
• The Microsoft.DataFactory resource provider registered in the subscription.

terraform-azurerm-data-factory-dataset-snowflake/
├── providers.tf    # required_version + the pinned azurerm; no provider block
├── variables.tf    # 12 typed inputs, 14 validations
├── main.tf         # the keystone, the numeric analysis, two dynamic blocks
├── outputs.tf      # 64 outputs; id first
├── README.md       # this file
├── SCOPE.md        # the cross-module contract
├── LICENSE         # MIT
└── .gitignore

provider "azurerm" {
  features {}
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_name = "ANALYTICS"
  table_name  = "ORDERS"
}

ℹ️ The schema has its own argument here. Do not write ANALYTICS.ORDERS into table_name — that is the shape three siblings need and this resource does not.

Consumes

Emits (selected — 64 in total)

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_name = "ANALYTICS"
  table_name  = "ORDERS"
}

module "resolves_to_nothing" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_empty"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name
  # no table, no parameters
}

check "dataset_points_somewhere" {
  assert {
    condition     = !module.resolves_to_nothing.describes_no_specific_table
    error_message = "ds_empty names no table and exposes no parameters; no pipeline can resolve it."
  }
}

# ❌ NOT ACCEPTED — twice over
module "copied_from_a_sibling" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_copied"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_column = [
    { name = "ORDER_ID", type = "Int64", description = "Primary key." },
  ]
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name
  schema_name         = "ANALYTICS"
  table_name          = "ORDERS"

  schema_column = [
    { name = "ORDER_ID", type = "NUMBER", precision = 38, scale = 0 },
    { name = "AMOUNT", type = "NUMBER", precision = 18, scale = 2 },
    { name = "CUSTOMER", type = "VARCHAR" },
    { name = "PLACED_AT", type = "TIMESTAMP_NTZ" },
  ]
}

module "silently_wrong" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_silently_wrong"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_column = [
    { name = "AMOUNT", type = "NUMBER", precision = 2, scale = 9 },
  ]
}

check "numeric_columns_are_coherent" {
  assert {
    condition     = !module.silently_wrong.numeric_column_definitions_look_wrong
    error_message = "A column's scale exceeds its precision; Snowflake will reject it at run time."
  }
}

module "harmless_but_odd" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_odd"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_column = [
    { name = "CUSTOMER", type = "VARCHAR", precision = 5 },
  ]
}

output "reported" {
  value = module.harmless_but_odd.columns_with_precision_on_a_non_numeric_type # ["CUSTOMER"]
}

module "default_precision" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_default_precision"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_column = [
    { name = "AMOUNT", type = "NUMBER", scale = 4 },
  ]
}

module "wrong_shape" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_wrong_shape"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  table_name = "ANALYTICS.ORDERS" # the schema belongs in schema_name
}

check "table_name_is_not_qualified" {
  assert {
    condition     = !module.wrong_shape.table_name_looks_qualified
    error_message = "table_name contains a dot; on this resource the schema belongs in schema_name."
  }
}

module "lower_case" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_lower"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_name = "analytics"
  table_name  = "orders"
}

output "case_check" {
  value = {
    schema = module.lower_case.schema_name_is_lower_case # true
    table  = module.lower_case.table_name_is_lower_case  # true
  }
}

module "any_table" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_any_table"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  parameters = {
    schemaName = "ANALYTICS"
    tableName  = ""
  }
}

locals {
  tables = {
    orders    = { schema = "ANALYTICS", table = "ORDERS" }
    customers = { schema = "ANALYTICS", table = "CUSTOMERS" }
    ledger    = { schema = "FINANCE", table = "LEDGER" }
  }
}

module "tables" {
  for_each = local.tables

  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_${each.key}"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name
  schema_name         = each.value.schema
  table_name          = each.value.table

  folder = "bronze/snowflake"
}

check "no_table_name_is_qualified" {
  assert {
    condition     = alltrue([for m in module.tables : !m.table_name_looks_qualified])
    error_message = "A table_name contains a dot; the schema belongs in schema_name."
  }
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name

  schema_name = "ANALYTICS"        # WHERE THE TABLE IS, in Snowflake
  folder      = "bronze/snowflake" # where the DATASET appears, in the portal
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name
  schema_name         = "ANALYTICS"
  table_name          = "ORDERS"

  description = "Order header rows, read hourly by the curated pipeline."
  annotations = ["bronze", "hourly"]

  additional_properties = {
    "structure" = "[]"
  }
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = var.data_factory_id
  linked_service_name = var.snowflake_linked_service_name
  table_name          = "ORDERS"

  timeouts = {
    create = "45m"
    read   = "10m"
    update = "45m"
    delete = "45m"
  }
}

provider "azurerm" {
  features {}
}

module "rg" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"

  name     = "rg-analytics-eastus"
  location = "eastus"
}

module "adf" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory.git?ref=v1.0.0"

  name                = "adf-analytics-eastus"
  resource_group_name = module.rg.name
  location            = module.rg.location

  identity = {
    type = "SystemAssigned"
  }
}

# No linked-service module exists in this suite yet, so the linked service is
# declared directly. Its connection string carries the Snowflake credential and
# chooses the ACCOUNT, WAREHOUSE and DATABASE -- keep it in a secret store and
# pass a reference; never inline it.
resource "azurerm_data_factory_linked_service_snowflake" "warehouse" {
  name            = "ls_snowflake_analytics"
  data_factory_id = module.adf.id

  connection_string = var.snowflake_connection_string # sensitive; supplied out of band
}

module "orders" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-data-factory-dataset-snowflake.git?ref=v1.0.0"

  name                = "ds_orders"
  data_factory_id     = module.adf.id
  linked_service_name = azurerm_data_factory_linked_service_snowflake.warehouse.name

  schema_name = "ANALYTICS"
  table_name  = "ORDERS"
  folder      = "bronze/snowflake"

  schema_column = [
    { name = "ORDER_ID", type = "NUMBER", precision = 38, scale = 0 },
    { name = "AMOUNT", type = "NUMBER", precision = 18, scale = 2 },
    { name = "PLACED_AT", type = "TIMESTAMP_NTZ" },
  ]

  annotations = ["bronze", "hourly"]
}

check "wiring_is_consistent" {
  assert {
    condition     = !module.orders.numeric_column_definitions_look_wrong
    error_message = "A numeric column definition is incoherent."
  }
  assert {
    condition     = !module.orders.table_name_looks_qualified
    error_message = "table_name contains a dot; the schema belongs in schema_name."
  }
  assert {
    condition     = !module.orders.describes_no_specific_table
    error_message = "The dataset resolves to nothing."
  }
}

output "dataset_id" {
  value = module.orders.id
}

output "reads_from" {
  value = module.orders.qualified_table_name
}

Required: name, data_factory_id, linked_service_name. What it points at: schema_name, table_name, parameters. Shape: schema_column ({name, type, precision, scale}), additional_properties. Metadata: description, folder, annotations. Tail: timeouts. There is no tags variable — the resource supports none.

No output is sensitive, and none can be.

Two column vocabularies that share nothing. Ten of the twelve Data Factory dataset resources accept fifteen Data Factory types — Byte, Int64, String and so on. This one accepts thirty-two Snowflake SQL types — NUMBER, VARCHAR, GEOGRAPHY. Not one value appears in both sets, and both are compared case-sensitively, so a column list is refused in whichever direction it is copied. The offline harness proves this exhaustively rather than asserting it: every one of the 47 values is driven against both this module and a Data Factory-typed sibling, and each must be accepted by exactly one side.

The block is a different shape as well as a different vocabulary. There is no description field here — precision and scale sit in its place. That makes a copied column a type error at parse time, which is the better failure: it names the attribute rather than producing a confusing validation message.

Four type values are not plain words. DOUBLE PRECISION has a space; three TIMESTAMP_* variants have underscores. Any list rebuilt by pattern-matching letters and digits loses exactly those four and looks complete — which is a mistake this library made once while extracting the set, and the reason the module carries the list verbatim.

precision and scale are barely validated. The provider applies IntAtLeast(0) to each and never compares them to each other or to the type. So scale > precision plans cleanly and Snowflake refuses it at run time; a precision on a VARCHAR is accepted and ignored; a scale with no precision takes Snowflake's default. The module reports all three separately, because only the first is certainly an error — collapsing them would overstate the other two.

The schema has its own argument here, and does not on three siblings. A dotted table_name is the correct shape on the SQL Server Table, MySQL and PostgreSQL datasets and the wrong one here, where it reaches Snowflake as a single identifier. qualified_table_name is emitted for display and documented as such, because writing it back into table_name is exactly the mistake.

Identifier case is reported, not corrected. Snowflake folds unquoted identifiers to upper case, so public becomes PUBLIC — unless the object was created with quotes, in which case the lower-case value is right. Normalising would silently break the quoted case, which is the harder one to diagnose.

Only identity forces replacement, and the linked service is not force-new — so moving a dataset to a different Snowflake account, warehouse and database is an in-place attribute change no destroy-scan will surface.

🔒 There is no risky toggle to close here: the resource holds no credential and touches no data. The exposures are the delete, the numeric definitions the provider does not check, and the Snowflake reach the linked service confers — the module documents the first and reports the second.

terraform init -backend=false
terraform validate
terraform fmt -check

Pin the module with ?ref=v1.0.0 — never a branch. This library is plan-only: a human applies from CI.

terraform plan and the module's own validation {} blocks cover, offline and without credentials — note it is plan and not terraform validate, which through a module call evaluates no variable values and reports success:

• all 32 Snowflake types accepted at their exact casing, including the space and underscore values;
• every Data Factory type refused, with the disjointness named in the message;
• precision and scale refused when negative or fractional, and accepted at zero;
• the anchored Resource-ID shape of data_factory_id, and the refusal of an ID passed as linked_service_name;
• every derived flag — the three numeric mismatches and their roll-up, table_name_looks_qualified, both identifier-case flags and describes_no_specific_table — each with one fixture per branch through terraform console.

The disjointness is executed, not asserted. The harness drives all 47 type values against both this module and a Data Factory-typed sibling — 94 module-runs — and requires each value to be accepted by exactly one side.

Only a real plan or apply can tell you whether the linked service exists, whether the table exists, whether the identifier case matches, or whether a numeric definition Snowflake dislikes will be rejected.

dataset_id = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-analytics-eastus/providers/Microsoft.DataFactory/factories/adf-analytics-eastus/datasets/ds_orders"
reads_from = "ANALYTICS.ORDERS"

linked_service_name                   = "ls_snowflake_analytics"
has_schema_name                       = true
describes_no_specific_table           = false
table_name_looks_qualified            = false
schema_name_is_lower_case             = false
schema_column_names                   = ["ORDER_ID", "AMOUNT", "PLACED_AT"]
columns_with_precision                = ["ORDER_ID", "AMOUNT"]
columns_with_scale_exceeding_precision = []
numeric_column_definitions_look_wrong = false
force_new_fields                      = ["name", "data_factory_id"]

• azurerm_data_factory_dataset_snowflake
• azurerm_data_factory_dataset_mysql — a Data Factory-typed sibling
• azurerm_data_factory_dataset_binary — no schema_column block at all
• Azure Data Factory datasets
• Snowflake connector
• SCOPE.md — this module's cross-module contract
• Sibling modules: terraform-azurerm-data-factory, terraform-azurerm-data-factory-dataset-mysql, terraform-azurerm-data-factory-dataset-postgresql, terraform-azurerm-data-factory-dataset-sql-server-table

💙 "Infrastructure as Code should be standardized, consistent, and secure."