Skip to content

Azure Blob ObjectStore Module

This OpenTofu module provisions Standard Azure Storage Accounts (General Purpose v2) for Blob object storage, with a mirrored Azure Table for every container.

It uses a "sharding" strategy: the top-level key of the shards map becomes one Storage Account, and the nested map under it names the private Blob containers inside that account. Sharding increases aggregate throughput and bounds the blast radius of a single account's limits. Each container also gets a matching Azure Table in the same account, named after the container with its dashes stripped, so blob and table access share one set of network rules and role assignments.

Features

  • Standard Performance: Standard tier with LRS replication (General Purpose v2) for cost-effective object storage.
  • Blob Versioning: Enabled on every account, so an overwrite or soft-deleted blob stays recoverable.
  • Sharding: Each top-level shards key becomes a dedicated Storage Account, and each account holds as many containers as its nested map lists — subject only to the 24-character account name budget.
  • Declared Containers: Nested map keys become private containers. An empty shard map synthesizes a single container named data.
  • Table Storage Mirroring: Every container gets an azurerm_storage_table named after it with all - stripped, created in the same account.
  • Plan-Time Table Validation: Invalid or colliding table names abort the plan via a terraform_data precondition rather than surfacing as an opaque Azure API error at apply time.
  • Managed Identity Access: Grants Storage Blob Data Contributor and Storage Table Data Contributor on every account to user_managed_identity_principal_id, so workloads authenticate without connection strings.
  • Network Security: With enable_firewall = true (the default) the accounts set default_action = "Deny", admit allowed_subnet_ids via virtual network rules and allowed_ips via IP rules, and bypass AzureServices. With it off, the network rules block opens to Allow with no restrictions.
  • HTTPS Enforcement: https_traffic_only_enabled = true and min_tls_version = "TLS1_2" on every account.
  • Firewall Drift Tolerance: network_rules[0].ip_rules is in ignore_changes, so the temporary runner-IP whitelists that CI/CD applies to these accounts do not show up as drift.

Requirements

No requirements.

Providers

Name Version
azurerm 4.80.0
terraform n/a

Modules

No modules.

Resources

Name Type
azurerm_role_assignment.umi_storage_access resource
azurerm_role_assignment.umi_table_access resource
azurerm_storage_account.object resource
azurerm_storage_container.containers resource
azurerm_storage_table.tables resource
terraform_data.table_name_check resource

Inputs

Name Description Type Default Required
allowed_ips List of ips to whitelist list(string) [] no
allowed_subnet_ids The IDs of the subnets that can access the object store. list(string) n/a yes
base_name [d] The base name prefix used for resources. string n/a yes
enable_firewall Enables storage account firewall bool true no
location [d] The location where resources will be created. string n/a yes
resource_group_name [d] The name of the resource group. string n/a yes
shards [d] A map of shard keys (each will get its own storage account) map(map(object({}))) n/a yes
suffix [d] A suffix to append to the resource name. string n/a yes
tags A map of tags to apply to the resource. map(string) n/a yes
user_managed_identity_principal_id [d] The principal ID of the user managed identity (required for role assignments). string n/a yes

[d] Destructive: changing this input forces one or more resources to be destroyed and recreated (an OpenTofu/Terraform replacement) rather than updated in place.

Outputs

Name Description
storage_accounts Map of object store details
tables Map of tables created per shard, with IDs for downstream chaining

Example Usage

module "object_store" {
  source = "oci://acrmerkalisdist0c66.azurecr.io/modules/azure/objectstore?tag=<module-version>"

  base_name           = "myapp"
  suffix              = random_id.project_prefix.hex
  resource_group_name = azurerm_resource_group.env.name
  location            = azurerm_resource_group.env.location

  # Required for the virtual network rules
  allowed_subnet_ids = [
    module.networking.created_subnets["storage"].id,
    module.networking.created_subnets["compute"].id
  ]

  # Required: principal of the identity that workloads use to reach the data
  user_managed_identity_principal_id = azurerm_user_assigned_identity.app_identity.principal_id

  # Shards; nested keys are the containers within each account
  shards = {
    "sn1" = {
      "hot-nodes-1" = {}
    }
    "sb1" = {
      "hot-blocks-1" = {}
    }
    "sb2" = {} # empty -> a single container named "data"
  }

  # The runner's IP must be present for plan/apply to reach the accounts
  allowed_ips = var.allowed_ips
  tags        = azurerm_resource_group.env.tags
}

Shards Variable Structure

shards is a map of shard keys to a map of container names. The shard key selects the Storage Account, the inner keys become containers within it:

shards = {
  "sb1" = {
    "hot-blocks-1" = {}
    "audit"        = {}
  }
  "sb2" = {} # no containers declared -> synthesizes one container named "data"
}

The inner value is currently always an empty object — container names carry all the information the module needs, and per-container settings have nowhere to go yet.

To add a container to an existing shard, add the key: the new container and its mirrored table are created on the next apply. Removing a key destroys both. Renaming a key replaces the container, which is destructive.

Notes

1. Table Storage Mirroring

Every container (including the synthesized default data container) gets a matching azurerm_storage_table named after the container with all - stripped:

Container name Table name
data data
my-logs mylogs

Tables are created in the same storage account (shard) as their container, so no shard prefix is added.

Azure table names must start with a letter and be 3-63 alphanumeric characters. This module strips - and validates the result against that rule at plan time, so a container name that would produce an invalid table name (e.g. starts with a digit or is too short) fails fast with a precondition error — before anything is applied.

Collisions: If two containers within the same shard strip to the same table name (e.g. my-data and mydata), the plan fails with a precondition error listing the offending shard and table name. Rename the containers to resolve.

Existing environments: New tables are empty. If a same-named table already exists in a storage account (e.g. created manually), the apply will conflict. Import the existing table first:

tofu import 'module.object_store.azurerm_storage_table.tables["<shard_key>-<container_name>"]' 'https://<storage-account>.table.core.windows.net/Tables('\''<table-name>'\'')'

2. Naming Limitations

Azure Storage Account names are strictly limited to 24 characters and must be lowercase alphanumeric. This module constructs names using the pattern:

sa + base_name + suffix + shard_key

Only the sa + base_name + suffix prefix is scrubbed to lowercase alphanumerics; the shard_key is concatenated verbatim, so shard keys must themselves be lowercase alphanumeric — sb01 is fine, hot-blocks-01 produces an invalid name.

Warning: You must ensure your inputs are short enough. - Prefix: sa (2 chars) - Base: myapp (5 chars) - Suffix: hex id (4 chars) - Key: sb01 (4 chars) - Total: 15/24 chars (Safe)

An over-long name is silently truncated by substr(..., 0, 24) rather than rejected at plan time. Two shards whose keys share a prefix past the 24th character therefore collapse onto the same account name and the apply fails with an "already exists" conflict — keep the distinctive part of each shard key within the first 24 characters.

3. Secure Transfer Requirements

All storage accounts are configured to: - Require HTTPS: HTTP traffic is rejected (https_traffic_only_enabled = true) - Use TLS 1.2+: Minimum TLS version is set to 1.2 for encryption in transit

4. Managed Identity Access

user_managed_identity_principal_id is a required input and creates two role assignments per shard: - Storage Blob Data Contributor for the container/blob data - Storage Table Data Contributor for the mirrored tables - This enables passwordless authentication using Entra ID. - The identity must exist in the same or a trusted Entra tenant.

The storage_accounts output still exposes primary_conn_string and primary_access_key (the whole map is marked sensitive). Prefer the identity-based access above; the keys are there for bootstrap and tooling scenarios, and nothing in this repo currently reads them.

5. Network Access

With the default enable_firewall = true, the module sets default_action = "Deny" on the network rules: - Access is only allowed from allowed_subnet_ids. - Access is also allowed for trusted AzureServices via bypass. - Access via Managed Identity still requires the workload to be inside the allowed subnet or otherwise have appropriate network routing — the identity authorizes the request, the firewall decides the path. - For the Azure Portal or an on-prem client, add its egress IP to allowed_ips.

Because network isolation is enforced by denying access to all non-whitelisted IPs, plan and apply fail (during the survey phase) with 403 Forbidden if the runner's IP address is not contained in allowed_ips. The CI/CD workflows whitelist the runner IP temporarily and rely on ignore_changes over network_rules[0].ip_rules so those one-off additions never appear as drift.

Setting enable_firewall = false publishes the accounts to the whole internet (default_action = "Allow" with no subnet, IP, or bypass rules) — intended only for environments (e.g., temporary performance test environments) where isolation is either not needed or handled elsewhere.

Available versions

  • v0.9.1