> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server.

# Terraform Users Guide for ADC

## Table of Contents

1. [Introduction](#introduction)
2. [Repository Overview](#repository-overview)
3. [Repository Structure](#repository-structure)
4. [Naming Conventions](#naming-conventions)
5. [Networking Layer](#networking-layer)
6. [Shared Services Layer](#shared-services-layer)
7. [Application Layer](#application-layer)
8. [Identity and Permissions](#identity-and-permissions)
9. [Application Registrations](#application-registrations)
10. [Code Examples](#code-examples)
11. [Best Practices](#best-practices)
12. [Troubleshooting](#troubleshooting)

---

## Introduction

This repository contains Terraform Infrastructure as Code (IaC) for deploying Azure Landing Zone components for the ADC (Application Development Center) environment. It leverages **Spacelift** modules from the private registry `us.spacelift.io/claconnect`.

### Owner Information

* **Owner**: Aaron Alex
* **Ops Team**: Data Operations
* **Support Contact**: Jon Cormack

### Purpose

This repository provisions and manages Azure resources across multiple environments (dev, qa, uat, prd) and regions (eastus2, centralus) using a three-layer architecture pattern.

---

## Repository Overview

### Architecture Layers

The repository is organized into three distinct layers:

1. **Networking Layer** (`networking/`)
   * Virtual networks and subnets
   * Network security configuration
   * Network watchers

2. **Shared Services Layer** (`shared_services/`)
   * Container Registry (ACR)
   * Shared Key Vaults
   * Shared Storage Accounts
   * Subscription budgets

3. **Application Layer** (`app/`)
   * AKS Clusters
   * SQL Servers and Databases
   * Redis Cache
   * Key Vaults (app-specific)
   * Storage Accounts
   * Service Bus
   * App Configuration
   * User Assigned Managed Identities
   * Application Registrations

### Module Source Pattern

All modules are sourced from the Spacelift private registry:

```hcl
module "example" {
  source  = "us.spacelift.io/claconnect/<module_name>/<provider>"
  version = "x.y.z"
  # ... configuration
}
```

---

## Repository Structure

```
az-adc-lz-tf/
├── README.md
├── app/
│   ├── *.tf                        # Resource definitions
│   └── config/
│       ├── dev.eastus2.auto.tfvars # Environment-specific variables
│       ├── qa.eastus2.auto.tfvars
│       ├── uat.eastus2.auto.tfvars
│       ├── prd.eastus2.auto.tfvars
│       └── sbx.eastus2.auto.tfvars
├── networking/
│   ├── *.tf
│   └── config/
│       ├── dev.eastus2.auto.tfvars
│       ├── dev.centralus.auto.tfvars
│       └── ...
└── shared_services/
    ├── *.tf
    └── config/
        ├── dev.eastus2.auto.tfvars
        └── ...
```

### Key Files in Application Layer

| File                  | Purpose                                       |
| --------------------- | --------------------------------------------- |
| `main.tf`             | Resource group definition                     |
| `provider.tf`         | Provider configuration                        |
| `variables.tf`        | Variable declarations (1100+ lines)           |
| `locals.tf`           | Local values and naming logic                 |
| `data.tf`             | Data source queries                           |
| `output.tf`           | Output values                                 |
| `aks_cluster.tf`      | AKS cluster and node pools                    |
| `database.tf`         | SQL servers and databases                     |
| `redis.tf`            | Redis cache instances                         |
| `kv.tf` & `kv_02.tf`  | Key Vault configurations                      |
| `storage.tf`          | Storage accounts and containers               |
| `servicebus.tf`       | Service Bus namespaces, topics, subscriptions |
| `app_config.tf`       | App Configuration service                     |
| `managed_identity.tf` | User Assigned Managed Identities              |
| `app_reg.tf`          | Azure AD App Registrations                    |
| `roles.tf`            | Role assignments and permissions              |
| `dns.tf`              | DNS configuration                             |
| `certs.tf`            | Certificate management                        |
| `monitoring.tf`       | Monitoring and alerts                         |

---

## Naming Conventions

The repository uses a consistent naming convention across all resources:

### Pattern

```
{environment}-{location}-{application}-{resource_type}-{suffix}
```

### Examples

* Resource Group: `dev-eus2-adc-app-rg-001`
* AKS Cluster: `dev-eus2-adc-aks-001`
* Key Vault: `dev-eus2-adc-app-kv-001`
* SQL Server: `dev-eus2-adc-sql-001`
* Storage Account: `deveus2adcinst001` (lowercase, no hyphens)

### Abbreviations

| Full Name                      | Abbreviation |
| ------------------------------ | ------------ |
| Development                    | dev          |
| Quality Assurance              | qa           |
| User Acceptance Testing        | uat          |
| Production                     | prd          |
| Sandbox                        | sbx          |
| East US 2                      | eus2         |
| Central US                     | cus          |
| Application Development Center | adc          |

### Naming Logic in `locals.tf`

All resource names are computed in `locals.tf`:

```hcl
locals {
  resource_group_name = "${var.environment_abbreviation}-${var.location_abbreviation}-${var.application_abbreviation}-app-rg-001"
  
  aks_cluster_name = "${var.environment_abbreviation}-${var.location_abbreviation}-${var.application_abbreviation}-aks-001"
  
  storage_account_names = { 
    for k, v in var.storage_accounts : 
    k => "${var.environment_abbreviation}${var.location_abbreviation}${var.application_abbreviation}${k}st001" 
  }
  
  key_vault_name = "${var.environment_abbreviation}-${var.location_abbreviation}-${var.application_abbreviation}-app-kv-001"
}
```

---

## Networking Layer

### Purpose

Provisions foundational networking infrastructure including VNets, subnets, and network security components.

### Components

* **Virtual Networks**: Address space allocation per environment
* **Subnets**: Segmentation for different workloads
  * Private Endpoints subnet
  * AKS Cluster subnet
  * AKS Load Balancer subnet
  * Function App subnet
* **Network Watcher**: Network monitoring and diagnostics

### Configuration Example

From `networking/config/dev.eastus2.auto.tfvars`:

```hcl
virtual_networks = {
  "dev-eus2-adc-vnet-001" = {
    vnet_address_space        = ["10.98.44.0/22"]
    dns_servers               = ["10.96.100.4"]
    log_retention_days        = 7
    traffic_analysis_interval = 60
  }
}

subnets = {
  "dev-eus2-adc-private-endpoints-snet-001" = {
    virtual_network_name                          = "dev-eus2-adc-vnet-001"
    address_prefixes                              = ["10.98.44.0/24"]
    service_endpoints                             = ["Microsoft.Storage", "Microsoft.KeyVault"]
    private_endpoint_network_policies             = "Enabled"
    private_link_service_network_policies_enabled = true
  },
  "dev-eus2-adc-aks-cluster-snet-001" = {
    virtual_network_name                          = "dev-eus2-adc-vnet-001"
    address_prefixes                              = ["10.98.45.0/24"]
    private_endpoint_network_policies             = "Enabled"
    private_link_service_network_policies_enabled = true
  }
}
```

### Resource Modules Used

* `claconnect/resource_group/azurerm`
* `claconnect/network_watcher/azurerm`
* `claconnect/virtual_network/azurerm`
* `claconnect/subnets/azurerm`

---

## Shared Services Layer

### Purpose

Deploys resources shared across multiple applications within the environment.

### Components

* **Container Registry (ACR)**: Docker image storage
* **Key Vaults**: Shared secrets management
* **Storage Accounts**: Shared storage
* **Subscription Budgets**: Cost management

### Configuration Structure

From `shared_services/config/dev.eastus2.auto.tfvars`:

```hcl
resource_group_name  = "dev-eus2-adc-shared-services-rg-01"

keyvaults = {
  # Empty by default - add as needed
}

storage_accounts = {
  # Empty by default - add as needed
}

action_groups = {
  "dev-eus2-adc-service-health-ag-001" = {
    action_group_short_name = "svc-health"
    email_receivers = [
      {
        name          = "Aaron Alex"
        email_address = "aaron.alex@claconnect.com"
      }
    ]
  }
}
```

### Key Vault Configuration Example

Key Vaults use random ID suffixes for global uniqueness:

```hcl
module "key_vault" {
  source   = "us.spacelift.io/claconnect/key_vault/azurerm"
  version  = "1.0.2"
  for_each = var.keyvaults

  key_vault_name                = "${each.key}-${module.random_id_keyvault.hex}"
  key_vault_resource_group_name = module.resource_group.name
  location                      = var.location
  
  enabled_for_template_deployment = each.value.enabled_for_template_deployment
  enabled_for_deployment          = each.value.enabled_for_deployment
  enable_rbac_authorization       = each.value.enable_rbac_authorization
  network_allowed_ip_rules        = var.nat_gateway_ips
  key_vault_public_access_enabled = each.value.public_access_enabled
  
  tags = var.tags
}
```

---

## Application Layer

### Overview

The application layer contains the bulk of the infrastructure, including compute, storage, databases, and identity resources.

---

### AKS Cluster

#### Configuration Files

* `aks_cluster.tf`: Cluster and node pool definitions

#### Module Configuration

```hcl
module "aks_cluster" {
  source  = "us.spacelift.io/claconnect/aks_cluster/azurerm"
  version = "1.1.16"

  resource_group_name                            = module.resource_group.name
  location                                       = var.location
  cluster_name                                   = local.aks_cluster_name
  kubernetes_version                             = var.aks_kubernetes_version
  automatic_upgrade_channel                      = var.aks_automatic_upgrade_channel
  private_cluster_enabled                        = var.aks_private_cluster_enabled
  private_dns_zone_id                            = var.aks_private_cluster_enabled ? local.aks_private_dns_zone_id : null
  
  # System Node Pool
  node_pool_name                                 = "system"
  node_pool_vm_size                              = var.aks_system_np_vm_size
  node_pool_os_disk_size_gb                      = var.aks_system_np_os_disk_size_gb
  node_pool_max_count                            = var.aks_system_np_max_count
  node_pool_min_count                            = var.aks_system_np_min_count
  node_pool_vnet_subnet_id                       = data.azurerm_subnet.aks_subnet.id
  
  # Identity
  identity_id                                    = data.azurerm_user_assigned_identity.aks_uami.id
  identity_clinet_id                             = data.azurerm_user_assigned_identity.aks_uami.client_id
  identity_object_id                             = data.azurerm_user_assigned_identity.aks_uami.principal_id
  
  # Networking
  pod_cidr                                       = "192.168.0.0/16"
  
  # Features
  service_mesh_enabled                           = var.aks_service_mesh_enabled
  service_mesh_mode                              = var.aks_service_mesh_mode
  image_cleaner_interval_hours                   = var.aks_image_cleaner_interval_hours
  
  tags = var.tags
}

module "aks_user_nodepool" {
  source  = "us.spacelift.io/claconnect/aks_nodepool/azurerm"
  version = "1.0.4"

  nodepool_name      = "userpool"
  aks_cluster_id     = module.aks_cluster.id
  kubernetes_version = var.aks_kubernetes_version
  vnet_subnet_id     = data.azurerm_subnet.aks_subnet.id
  vm_size            = var.aks_np_vm_size
  max_pods           = var.aks_np_max_pods
  min_count          = var.aks_np_min_count
  max_count          = var.aks_np_max_count
  node_labels        = var.aks_node_labels
}
```

#### Maintenance Windows

Configure maintenance windows in tfvars:

```hcl
maintenance_window_auto_upgrade = {
  frequency   = "Weekly"
  interval    = 1
  duration    = 4
  day_of_week = "Sunday"
  start_date  = "2025-10-29T00:00:00Z"
  start_time  = "00:00"
  utc_offset  = "-06:00"
}
```

---

### SQL Server and Databases

#### Configuration Files

* `database.tf`: SQL server and database definitions

#### SQL Server Module

```hcl
module "sql_server" {
  source  = "us.spacelift.io/claconnect/sql_server/azurerm"
  version = "~> 1.1.0"

  name                                             = local.sql_server_name
  resource_group_name                              = module.resource_group.name
  location                                         = var.location
  sql_version                                      = var.sql_server_version
  
  # Azure AD Authentication Only
  administrator_login                              = null
  administrator_login_password                     = null
  azuread_administrator_azread_authentication_only = true
  azuread_administrator_login_username             = var.sql_server_azuread_administrator_login_username
  azuread_administrator_object_id                  = var.sql_server_azuread_administrator_object_id
  
  public_network_access_enabled                    = var.sql_server_public_network_access_enabled
  tags                                             = var.tags
}

module "sql_server_private_endpoint" {
  source  = "us.spacelift.io/claconnect/private_endpoint/azurerm"
  version = "1.0.4"

  name                  = "${local.sql_server_name}-pe"
  location              = var.location
  resource_group_name   = module.resource_group.name
  subnet_id             = data.azurerm_subnet.pe_subnet.id
  tags                  = var.tags
  private_dns_zone_name = local.db_private_dns_zone_name
  private_dns_zone_ids  = [local.db_private_dns_zone_id]
  
  private_service_connection = {
    name                           = "${local.sql_server_name}-pe"
    private_connection_resource_id = module.sql_server.id
    subresource_names              = ["sqlServer"]
    is_manual_connection           = false
  }
}
```

#### SQL Database Module

```hcl
module "sql_database_autoextract" {
  source  = "us.spacelift.io/claconnect/sql_database/azurerm"
  version = "1.0.5"

  name                 = local.sql_database_autoextract_name
  sku_name             = var.sql_database_autoextract_sku_name
  server_id            = module.sql_server.id
  geo_backup_enabled   = var.sql_database_autoextract_geo_backup_enabled
  storage_account_type = var.sql_database_autoextract_storage_account_type
  zone_redundant       = var.sql_database_autoextract_zone_redundant
  
  # Serverless Configuration
  license_type         = null
  min_capacity         = 0.5
  auto_pause_delay     = var.sql_database_autoextract_auto_pause_delay
  
  tags = var.tags
}
```

#### Auditing Configuration

```hcl
module "sql_server_extended_auditing_policy" {
  source  = "us.spacelift.io/claconnect/mssql_server_extended_auditing_policy/azurerm"
  version = "1.0.0"

  sql_server_id     = module.sql_server.id
  enabled           = true
  retention_in_days = 0

  audit_actions_and_groups = [
    "SUCCESSFUL_DATABASE_AUTHENTICATION_GROUP",
    "FAILED_DATABASE_AUTHENTICATION_GROUP",
    "BATCH_COMPLETED_GROUP"
  ]
}
```

---

### Redis Cache

#### Configuration Files

* `redis.tf`: Redis cache instances

#### Redis Module

```hcl
module "axcess_redis_cache" {
  source  = "us.spacelift.io/claconnect/redis_cache/azurerm"
  version = "1.0.2"

  location            = var.location
  name                = local.axcess_redis_server_name
  resource_group_name = module.resource_group.name

  # Entra ID Authentication (no access keys)
  access_keys_authentication_enabled = false
  redis_configuration = {
    active_directory_authentication_enabled = true
  }

  # SKU Configuration
  sku_name = var.axcess_redis_server_sku_name  # "Premium", "Standard", or "Basic"
  capacity = var.axcess_redis_server_capacity  # 0-6 depending on SKU
  family   = var.axcess_redis_server_family    # "C" (Basic/Standard) or "P" (Premium)
}

module "axcess_redis_cache_private_endpoint" {
  source  = "us.spacelift.io/claconnect/private_endpoint/azurerm"
  version = "1.0.4"

  name                  = "${local.axcess_redis_server_name}-pe"
  location              = var.location
  resource_group_name   = module.resource_group.name
  subnet_id             = data.azurerm_subnet.pe_subnet.id
  tags                  = var.tags
  private_dns_zone_name = local.redis_private_dns_zone_name
  private_dns_zone_ids  = [local.redis_private_dns_zone_id]
  
  private_service_connection = {
    name                           = "${local.axcess_redis_server_name}-pe"
    private_connection_resource_id = module.axcess_redis_cache.id
    subresource_names              = ["redisCache"]
    is_manual_connection           = false
  }
}
```

---

### Key Vault

#### Configuration Files

* `kv.tf`: Primary Key Vault
* `kv_02.tf`: Secondary Key Vault

#### Key Vault Module

```hcl
module "key_vault" {
  source  = "us.spacelift.io/claconnect/key_vault/azurerm"
  version = "1.0.3"

  key_vault_name                  = local.key_vault_name
  key_vault_resource_group_name   = local.resource_group_name
  location                        = var.location
  tags                            = var.tags
  
  key_vault_public_access_enabled = true
  enabled_for_deployment          = true
  rbac_authorization_enabled      = true
  
  # Network Security
  network_default_action          = "Deny"
  network_allowed_ip_rules        = var.ip_rules
  virtual_network_subnet_ids      = var.virtual_network_rule_ids
}

module "key_vault_private_endpoint" {
  source  = "us.spacelift.io/claconnect/private_endpoint/azurerm"
  version = "1.0.4"

  name                  = "${local.key_vault_name}-pe"
  location              = var.location
  resource_group_name   = module.resource_group.name
  subnet_id             = data.azurerm_subnet.pe_subnet.id
  tags                  = var.tags
  private_dns_zone_name = local.key_vault_private_dns_zone_name
  private_dns_zone_ids  = [local.key_vault_private_dns_zone_id]
  
  private_service_connection = {
    name                           = "${local.key_vault_name}-pe"
    private_connection_resource_id = module.key_vault.id
    subresource_names              = ["vault"]
    is_manual_connection           = false
  }

  depends_on = [module.key_vault]
}
```

#### Secrets Management

Secrets are defined in `locals.tf` and created via module:

```hcl
# In locals.tf
locals {
  kv_secrets = {
    "aks-api-ingress-ip"              = local.api_ingress_ip
    "fluxx-api-url"                   = contains(keys(var.app_service_certificates), "fluxx-api") ? replace(var.app_service_certificates["fluxx-api"].distinguished_name, "CN=", "") : ""
    "documents-ai-api-url"            = contains(keys(var.app_service_certificates), "documents-ai-api") ? replace(var.app_service_certificates["documents-ai-api"].distinguished_name, "CN=", "") : ""
  }
}

# In kv.tf
module "kv_secrets" {
  source  = "us.spacelift.io/claconnect/key_vault_secret/azurerm"
  version = "1.0.2"

  for_each = local.kv_secrets

  key_vault_id           = module.key_vault.id
  key_vault_secret_name  = each.key
  key_vault_secret_value = each.value
  tags                   = var.tags

  depends_on = [module.time_sleep, module.assign_deployment_uami_key_vault_secrets_officer]
}
```

---

### Storage Accounts

#### Configuration Files

* `storage.tf`: Storage account and container definitions

#### Storage Account Module

```hcl
module "storage_account" {
  source   = "us.spacelift.io/claconnect/storage_account/azurerm"
  version  = "1.0.10"
  for_each = var.storage_accounts

  name                          = local.storage_account_names[each.key]
  location                      = var.location
  resource_group_name           = module.resource_group.name
  account_replication_type      = each.value.replication_type
  network_bypass                = each.value.network_bypass
  is_hns_enabled                = each.value.is_hns_enabled
  public_network_access_enabled = each.value.public_access_enabled
  account_kind                  = each.value.account_kind
  network_default_action        = each.value.network_default_action
  shared_access_key_enabled     = true

  allowed_public_ips         = var.ip_rules
  virtual_network_subnet_ids = var.storage_virtual_network_subnet_ids
  tags                       = var.tags
}

module "storage_container" {
  source  = "us.spacelift.io/claconnect/storage_container/azurerm"
  version = "1.0.3"

  for_each = var.storage_account_containers

  container_name                    = each.key
  storage_account_id                = module.storage_account[each.value.storage_account_key].id
  access_type                       = each.value.access_type
  default_encryption_scope          = each.value.default_encryption_scope
  encryption_scope_override_enabled = each.value.encryption_scope_override_enabled
  metadata                          = each.value.metadata
  immutability_period               = each.value.immutability_period
}

module "storage_account_blob_private_endpoint" {
  source  = "us.spacelift.io/claconnect/private_endpoint/azurerm"
  version = "1.0.4"

  for_each = var.storage_accounts

  name                  = "${local.storage_account_names[each.key]}-pe"
  location              = var.location
  resource_group_name   = module.resource_group.name
  subnet_id             = data.azurerm_subnet.pe_subnet.id
  tags                  = var.tags
  private_dns_zone_name = local.blob_private_dns_zone_name
  private_dns_zone_ids  = [local.blob_private_dns_zone_id]
  
  private_service_connection = {
    name                           = "${local.storage_account_names[each.key]}-pe"
    private_connection_resource_id = module.storage_account[each.key].id
    subresource_names              = ["blob"]
    is_manual_connection           = false
  }
}
```

#### Variable Configuration Example

```hcl
storage_accounts = {
  "in" = {
    replication_type       = "LRS"
    network_bypass         = ["AzureServices"]
    is_hns_enabled         = false
    public_access_enabled  = true
    account_kind           = "StorageV2"
    network_default_action = "Deny"
  }
  "doc" = {
    replication_type       = "LRS"
    network_bypass         = ["AzureServices"]
    is_hns_enabled         = false
    public_access_enabled  = true
    account_kind           = "StorageV2"
    network_default_action = "Deny"
  }
}

storage_account_containers = {
  "input" = {
    storage_account_key                = "in"
    access_type                        = "private"
    default_encryption_scope           = "$account-encryption-key"
    encryption_scope_override_enabled  = false
    metadata                           = {}
    immutability_period                = null
  }
}
```

---

### Service Bus

#### Configuration Files

* `servicebus.tf`: Namespace, topics, and subscriptions

#### Service Bus Namespace

```hcl
module "servicebus_namespace" {
  source  = "us.spacelift.io/claconnect/servicebus_namespace/azurerm"
  version = "1.0.5"

  for_each = var.servicebus_namespaces

  name                                 = local.service_bus_namespace
  location                             = var.location
  resource_group_name                  = local.resource_group_name
  sku                                  = var.servicebus_namespace_sku
  capacity                             = var.servicebus_namespace_capacity
  premium_messaging_partitions         = var.servicebus_premium_messaging_partitions
  local_auth_enabled                   = each.value.local_auth_enabled
  public_network_access_enabled        = each.value.public_network_access_enabled
  minimum_tls_version                  = var.servicebus_namespace_minimum_tls_version
  tags                                 = var.tags
  identity_enabled                     = var.servicebus_identity_enabled
  ip_ranges                            = var.ip_rules
  trusted_services_allowed             = each.value.trusted_services_allowed
  subnet_id                            = data.azurerm_subnet.pe_subnet.id
  ignore_missing_vnet_service_endpoint = each.value.ignore_missing_vnet_service_endpoint
}
```

#### Service Bus Topics

```hcl
module "servicebus_topic" {
  source  = "us.spacelift.io/claconnect/servicebus_topic/azurerm"
  version = "1.0.0"

  for_each = { for key, value in var.servicebus_topics : key => value if contains(keys(var.servicebus_namespaces), value.namespace_key) }

  name                          = each.value.name
  namespace_id                  = module.servicebus_namespace[each.value.namespace_key].id
  status                        = var.servicebus_subscription_status
  batched_operations_enabled    = each.value.batched_operations_enabled
  express_enabled               = each.value.express_enabled
  partitioning_enabled          = var.servicebus_premium_messaging_partitions > 1 ? true : false
  max_message_size_in_kilobytes = var.servicebus_topic_max_message_size_in_kilobytes
  max_size_in_megabytes         = var.servicebus_topic_max_size_in_megabytes
  requires_duplicate_detection  = each.value.requires_duplicate_detection
  support_ordering              = each.value.support_ordering
}
```

#### Service Bus Subscriptions

```hcl
module "servicebus_subscription" {
  source  = "us.spacelift.io/claconnect/servicebus_subscription/azurerm"
  version = "1.0.1"

  for_each = { for key, value in var.servicebus_subscriptions : key => value if contains(keys(var.servicebus_topics), value.topic_key) }

  name                                      = each.value.name
  topic_id                                  = module.servicebus_topic[each.value.topic_key].id
  max_delivery_count                        = each.value.max_delivery_count
  dead_lettering_on_message_expiration      = each.value.dead_lettering_on_message_expiration
  dead_lettering_on_filter_evaluation_error = each.value.dead_lettering_on_filter_evaluation_error
  batched_operations_enabled                = each.value.batched_operations_enabled
  requires_session                          = each.value.requires_session
  status                                    = var.servicebus_subscription_status
  lock_duration                             = "PT1M"
}
```

#### Variable Configuration Example

```hcl
servicebus_namespaces = {
  "ns1" = {
    local_auth_enabled                   = false
    public_network_access_enabled        = true
    trusted_services_allowed             = true
    ignore_missing_vnet_service_endpoint = true
  }
}

servicebus_topics = {
  "vf" = {
    namespace_key                = "ns1"
    name                         = "validated-file"
    batched_operations_enabled   = false
    express_enabled              = false
    requires_duplicate_detection = false
    support_ordering             = false
  }
}

servicebus_subscriptions = {
  "cvf" = {
    topic_key                                 = "vf"
    name                                      = "copier-validated-file"
    dead_lettering_on_message_expiration      = false
    dead_lettering_on_filter_evaluation_error = false
    batched_operations_enabled                = false
    requires_session                          = false
    max_delivery_count                        = 1
    client_id                                 = ""
  }
}
```

---

### App Configuration

#### Configuration Files

* `app_config.tf`: App Configuration service

#### App Configuration Module

```hcl
module "app_configuration" {
  source  = "us.spacelift.io/claconnect/app_configuration/azurerm"
  version = "1.0.9"

  name                                  = local.app_config_name
  resource_group_name                   = module.resource_group.name
  location                              = var.location
  tags                                  = var.tags
  identity_id                           = module.user_assigned_identity["appcs"].id
  sku                                   = var.app_config_sku
  soft_delete_retention_days            = var.app_config_soft_delete_retention_days
  public_network_access                 = var.app_config_enable_public_network
  data_plane_proxy_authentication_mode  = var.app_config_authentication_mode
  local_auth_enabled                    = var.app_config_local_auth_enabled
}

module "app_configuration_private_endpoint" {
  source  = "us.spacelift.io/claconnect/private_endpoint/azurerm"
  version = "1.0.4"

  name                  = "${local.app_config_name}-pe"
  location              = var.location
  resource_group_name   = local.resource_group_name
  subnet_id             = data.azurerm_subnet.pe_subnet.id
  tags                  = var.tags
  private_dns_zone_name = local.appcon_private_dns_zone_name
  private_dns_zone_ids  = [local.appcon_private_dns_zone_id]

  private_service_connection = {
    name                           = "${local.app_config_name}-pe"
    private_connection_resource_id = module.app_configuration.id
    subresource_names              = ["configurationStores"]
    is_manual_connection           = false
  }
}
```

---

## Identity and Permissions

### User Assigned Managed Identities (UAMIs)

#### Configuration Files

* `managed_identity.tf`: UAMI definitions
* `roles.tf`: Role assignments

#### UAMI Module

```hcl
module "user_assigned_identity" {
  source  = "us.spacelift.io/claconnect/user_assigned_identity/azurerm"
  version = "1.1.1"

  for_each = var.user_assigned_identitys

  name                = local.uami_names[each.key]
  resource_group_name = module.resource_group.name
  location            = var.location
  
  federated_credentials = lookup(var.user_assigned_identitys[each.key], "federated_credentials", {}) != {} ? {
    for k, v in each.value.federated_credentials : k => {
      audience = ["api://AzureADTokenExchange"]
      issuer   = k == "github" ? "https://token.actions.githubusercontent.com" : k == "workload_identity" ? module.aks_cluster.oidc_issuer_url : v.issuer
      subject  = k == "github" ? "repo:CLADevOps/${v.repo}:environment:${var.environment_abbreviation}" : k == "workload_identity" ? "system:serviceaccount:${v.namespace}:${v.service_account_name}" : v.subject
    }
  } : {}
}
```

#### Federated Credentials

The module supports two types of federated credentials:

1. **GitHub Actions** - For CI/CD pipelines
2. **Workload Identity** - For AKS workloads

Example variable configuration:

```hcl
user_assigned_identitys = {
  "fluxx" = {
    federated_credentials = {
      github = {
        repo = "fluxx-api"
      }
      workload_identity = {
        namespace           = "fluxx"
        service_account_name = "fluxx-sa"
      }
    }
    key_vault_secrets_user = true
    service_bus_sender = true
    service_bus_reciever = true
  }
}
```

---

### Role Assignments

#### Permission Flags

User Assigned Managed Identities can be assigned permissions using boolean flags in the variable definition. The following flags are available:

| Flag                                      | Role                                        | Scope                     |
| ----------------------------------------- | ------------------------------------------- | ------------------------- |
| `aks_cluster_admin`                       | Azure Kubernetes Service RBAC Cluster Admin | AKS Cluster               |
| `service_bus_sender`                      | Azure Service Bus Data Sender               | Service Bus Namespace     |
| `service_bus_reciever`                    | Azure Service Bus Data Receiver             | Service Bus Namespace     |
| `input_storage_blob_data_contributor`     | Storage Blob Data Contributor               | Input Storage Account     |
| `input_storage_blob_data_reader`          | Storage Blob Data Reader                    | Input Storage Account     |
| `documents_storage_blob_data_contributor` | Storage Blob Data Contributor               | Documents Storage Account |
| `documents_storage_blob_data_reader`      | Storage Blob Data Reader                    | Documents Storage Account |
| `key_vault_secrets_user`                  | Key Vault Secrets User                      | Key Vault (primary)       |
| `key_vault_secrets_user_02`               | Key Vault Secrets User                      | Key Vault (secondary)     |
| `key_vault_secrets_officer`               | Key Vault Secrets Officer                   | Key Vault (secondary)     |
| `app_config_reader`                       | App Configuration Data Reader               | App Configuration         |
| `app_insights_publisher`                  | Monitoring Metrics Publisher                | App Insights              |

#### Role Assignment Pattern

All role assignments follow a consistent pattern using for\_each and lookup:

```hcl
module "assign_uamis_storage_blob_data_contributor_to_input" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  for_each = { for k, v in var.user_assigned_identitys : k => v if lookup(v, "input_storage_blob_data_contributor", false) == true }

  scope           = module.storage_account["in"].id
  role_definition = "Storage Blob Data Contributor"
  principal_id    = module.user_assigned_identity[each.key].principal_id
}

module "assign_uamis_key_vault_secrets_user" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  for_each = { for k, v in var.user_assigned_identitys : k => v if lookup(v, "key_vault_secrets_user", false) == true }

  scope           = module.key_vault.id
  role_definition = "Key Vault Secrets User"
  principal_id    = module.user_assigned_identity[each.key].principal_id
}

module "assign_uamis_service_bus_data_sender" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  for_each = { for k, v in var.user_assigned_identitys : k => v if lookup(v, "service_bus_sender", false) == true }

  scope           = module.servicebus_namespace["ns1"].id
  role_definition = "Azure Service Bus Data Sender"
  principal_id    = module.user_assigned_identity[each.key].principal_id
}
```

#### Pre-deployed Identity Role Assignments

Some identities are pre-deployed and referenced via data sources:

```hcl
module "assign_aks_uami_key_vault_secrets_officer" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  scope           = module.key_vault.id
  role_definition = "Key Vault Secrets Officer"
  principal_id    = data.azurerm_user_assigned_identity.aks_uami.principal_id
}

module "assign_deployment_uami_aks_cluster_admin" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  scope           = module.aks_cluster.id
  role_definition = "Azure Kubernetes Service RBAC Cluster Admin"
  principal_id    = data.azurerm_client_config.current.object_id
}
```

---

## Application Registrations

### Configuration Files

* `app_reg.tf`: App registration definitions

### Components

1. **UUID Generation**: Unique IDs for API URIs, permissions, and app roles
2. **Application Registration**: Azure AD app creation
3. **App Roles**: Custom roles for users and applications
4. **OAuth2 Permission Scopes**: Delegated permissions
5. **Role Assignments**: Mapping AAD groups and service principals to app roles

### Application Registration Module

```hcl
module "application_registrations" {
  source  = "us.spacelift.io/claconnect/application/azuread"
  version = "1.0.0"

  for_each = var.application_registrations

  display_name    = local.appreg_names[each.key]
  owners          = [data.azuread_client_config.current.object_id]
  identifier_uris = ["api://app-${module.app_reg_api_urls_uuids[each.key].random_ids}"]

  single_page_application = [
    {
      redirect_uris = each.value.spa_redictsion_uris
    }
  ]

  required_resource_access = [
    {
      resource_app_id = "00000003-0000-0000-c000-000000000000"  # Microsoft Graph
      resource_access = [
        {
          id   = "e1fe6dd8-ba31-4d61-89e7-88639da4683d"  # User.Read
          type = "Scope"
        }
      ]
    }
  ]

  app_roles = [
    for key, role in local.unique_app_roles : {
      id                   = module.app_reg_app_roles_uuids[key].random_ids
      display_name         = role.role_name
      description          = "Role for ${role.role_name}"
      value                = role.role_name
      allowed_member_types = role.allowed_member_types
    } if role.app_name == each.key
  ]

  api_oauth2_permission_scope = [
    for scope in each.value.oauth2_permission_scopes != null ? each.value.oauth2_permission_scopes : [] : {
      id                         = module.all_oauth2_permission_scopes_uuids["${each.key}.${scope.value}"].random_ids
      admin_consent_description  = scope.admin_consent_description
      admin_consent_display_name = scope.admin_consent_display_name
      type                       = scope.type
      user_consent_description   = scope.user_consent_description
      user_consent_display_name  = scope.user_consent_display_name
      value                      = scope.value
    }
  ]
}
```

### Variable Configuration Example

```hcl
application_registrations = {
  "fluxx" = {
    spa_redictsion_uris = [
      "https://oauth.pstmn.io/v1/callback",
      "https://oauth.pstmn.io/v1/browser-callback"
    ]
    
    app_roles = {
      "user" = {
        allowed_member_types = ["User"]
        group_names = ["AAD-FluxxUserDev"]
      }
      "admin" = {
        allowed_member_types = ["User"]
        group_names = ["AAD-FluxxAdminDev"]
      }
      "application" = {
        allowed_member_types = ["Application"]
        application_names = ["dev-eus2-adc-documents-ai-uami-001"]
      }
    }
    
    oauth2_permission_scopes = [
      {
        admin_consent_description  = "Allow Fluxx API"
        admin_consent_display_name = "Fluxx.Read"
        type                       = "User"
        user_consent_description   = "Allow Fluxx API"
        user_consent_display_name  = "Fluxx.Read"
        value                      = "Fluxx.Read"
      },
      {
        admin_consent_description  = "Allow Fluxx Files.Write"
        admin_consent_display_name = "Fluxx.Files.Write"
        type                       = "User"
        user_consent_description   = "Allow Fluxx Files.Write"
        user_consent_display_name  = "Allow Fluxx Files.Write"
        value                      = "Fluxx.Files.Write"
      }
    ]
  }
}
```

### App Role Assignments

App roles are assigned to Azure AD groups and service principals:

```hcl
module "assign_app_reg_roles" {
  source  = "us.spacelift.io/claconnect/app_role_assignment/azuread"
  version = "1.0.0"

  for_each = { for idx, assignment in local.flattened_app_roles : idx => assignment }

  resource_object_id = module.application_registrations[each.value.app_name].object_id
  principal_id       = each.value.principal_object_id
  app_role_id        = module.app_reg_app_roles_uuids["${each.value.app_name}.${each.value.role_name}"].random_ids
}
```

---

## Code Examples

### Example 1: Adding a New AKS Cluster

**Not applicable** - The repository uses a singleton pattern for AKS. To modify the existing cluster, update variables in the tfvars file.

To change node pool configuration:

```hcl
# In config/dev.eastus2.auto.tfvars
aks_kubernetes_version = "1.28.5"
aks_system_np_vm_size = "Standard_D4s_v3"
aks_system_np_min_count = 3
aks_system_np_max_count = 6
aks_np_vm_size = "Standard_D8s_v3"
aks_np_min_count = 2
aks_np_max_count = 10
```

---

### Example 2: Adding a New Redis Cache

#### Step 1: Add local name in `locals.tf`

```hcl
locals {
  # ... existing locals ...
  
  # Add new Redis server name
  new_app_redis_server_name = "${var.environment_abbreviation}-${var.location_abbreviation}-${var.application_abbreviation}-redis-002"
}
```

#### Step 2: Add variables in `variables.tf`

```hcl
variable "new_app_redis_server_sku_name" {
  description = "The SKU name of the new Redis server"
  type        = string
  default     = "Premium"
}

variable "new_app_redis_server_capacity" {
  description = "The capacity of the new Redis server"
  type        = number
  default     = 1
}

variable "new_app_redis_server_family" {
  description = "The family of the new Redis server"
  type        = string
  default     = "P"
}
```

#### Step 3: Add module in `redis.tf`

```hcl
module "new_app_redis_cache" {
  source  = "us.spacelift.io/claconnect/redis_cache/azurerm"
  version = "1.0.2"

  location            = var.location
  name                = local.new_app_redis_server_name
  resource_group_name = module.resource_group.name

  access_keys_authentication_enabled = false
  redis_configuration = {
    active_directory_authentication_enabled = true
  }

  sku_name = var.new_app_redis_server_sku_name
  capacity = var.new_app_redis_server_capacity
  family   = var.new_app_redis_server_family
}

module "new_app_redis_cache_private_endpoint" {
  source  = "us.spacelift.io/claconnect/private_endpoint/azurerm"
  version = "1.0.4"

  name                  = "${local.new_app_redis_server_name}-pe"
  location              = var.location
  resource_group_name   = module.resource_group.name
  subnet_id             = data.azurerm_subnet.pe_subnet.id
  tags                  = var.tags
  private_dns_zone_name = local.redis_private_dns_zone_name
  private_dns_zone_ids  = [local.redis_private_dns_zone_id]
  
  private_service_connection = {
    name                           = "${local.new_app_redis_server_name}-pe"
    private_connection_resource_id = module.new_app_redis_cache.id
    subresource_names              = ["redisCache"]
    is_manual_connection           = false
  }
}
```

#### Step 4: Set values in tfvars

```hcl
# In config/dev.eastus2.auto.tfvars
new_app_redis_server_sku_name = "Premium"
new_app_redis_server_capacity = 1
new_app_redis_server_family   = "P"
```

---

### Example 3: Adding a New SQL Database

#### Step 1: Add local name in `locals.tf`

```hcl
locals {
  # ... existing locals ...
  
  sql_database_new_app_name = "NewAppDatabase"
}
```

#### Step 2: Add variables in `variables.tf`

```hcl
variable "sql_database_new_app_sku_name" {
  description = "The SKU name for the new app database"
  type        = string
  default     = "GP_S_Gen5_2"
}

variable "sql_database_new_app_geo_backup_enabled" {
  description = "Enable geo backup for new app database"
  type        = bool
  default     = true
}

variable "sql_database_new_app_zone_redundant" {
  description = "Enable zone redundancy for new app database"
  type        = bool
  default     = false
}

variable "sql_database_new_app_auto_pause_delay" {
  description = "Auto pause delay in minutes for serverless SKU"
  type        = number
  default     = 60
}
```

#### Step 3: Add module in `database.tf`

```hcl
module "sql_database_new_app" {
  source  = "us.spacelift.io/claconnect/sql_database/azurerm"
  version = "1.0.5"

  name                 = local.sql_database_new_app_name
  sku_name             = var.sql_database_new_app_sku_name
  server_id            = module.sql_server.id
  geo_backup_enabled   = var.sql_database_new_app_geo_backup_enabled
  storage_account_type = "LRS"
  zone_redundant       = var.sql_database_new_app_zone_redundant
  license_type         = null
  min_capacity         = 0.5
  auto_pause_delay     = var.sql_database_new_app_auto_pause_delay
  tags                 = var.tags
}
```

#### Step 4: Set values in tfvars

```hcl
# In config/dev.eastus2.auto.tfvars
sql_database_new_app_sku_name = "GP_S_Gen5_2"
sql_database_new_app_geo_backup_enabled = true
sql_database_new_app_zone_redundant = false
sql_database_new_app_auto_pause_delay = 60
```

---

### Example 4: Adding a New UAMI with Permissions

#### Step 1: Add UAMI configuration in tfvars

```hcl
# In config/dev.eastus2.auto.tfvars
user_assigned_identitys = {
  # ... existing UAMIs ...
  
  "new-app" = {
    # Federated credentials for GitHub Actions
    federated_credentials = {
      github = {
        repo = "new-app-api"
      }
      workload_identity = {
        namespace           = "new-app"
        service_account_name = "new-app-sa"
      }
    }
    
    # Permission flags
    key_vault_secrets_user                = true
    app_config_reader                     = true
    service_bus_sender                    = true
    service_bus_reciever                  = true
    input_storage_blob_data_contributor   = true
    documents_storage_blob_data_reader    = true
    app_insights_publisher                = true
  }
}
```

#### Step 2: Apply Terraform

The UAMI will be created automatically with:

* Name: `dev-eus2-adc-new-app-uami-001`
* Federated credentials for GitHub Actions and AKS Workload Identity
* All specified role assignments

No changes to `.tf` files required - the existing module and role assignment patterns handle everything!

---

### Example 5: Adding Storage Account with Role Assignments

#### Step 1: Add storage account configuration in tfvars

```hcl
# In config/dev.eastus2.auto.tfvars
storage_accounts = {
  # ... existing storage accounts ...
  
  "newapp" = {
    replication_type       = "LRS"
    network_bypass         = ["AzureServices"]
    is_hns_enabled         = false
    public_access_enabled  = true
    account_kind           = "StorageV2"
    network_default_action = "Deny"
  }
}

storage_account_containers = {
  # ... existing containers ...
  
  "newapp-data" = {
    storage_account_key                = "newapp"
    access_type                        = "private"
    default_encryption_scope           = "$account-encryption-key"
    encryption_scope_override_enabled  = false
    metadata                           = {}
    immutability_period                = null
  }
}
```

#### Step 2: Grant permissions to UAMI

If you need a UAMI with specific permissions to this storage account, you would need to:

1. **Add new permission flags to role assignment modules** in `roles.tf`:

```hcl
module "assign_uamis_storage_blob_data_contributor_to_newapp" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  for_each = { for k, v in var.user_assigned_identitys : k => v if lookup(v, "newapp_storage_blob_data_contributor", false) == true }

  scope           = module.storage_account["newapp"].id
  role_definition = "Storage Blob Data Contributor"
  principal_id    = module.user_assigned_identity[each.key].principal_id
}
```

2. **Update UAMI configuration** in tfvars:

```hcl
user_assigned_identitys = {
  "new-app" = {
    # ... other settings ...
    newapp_storage_blob_data_contributor = true
  }
}
```

---

### Example 6: Adding Service Bus Topics and Subscriptions

#### Step 1: Add topic configuration in tfvars

```hcl
# In config/dev.eastus2.auto.tfvars
servicebus_topics = {
  # ... existing topics ...
  
  "na" = {
    namespace_key                = "ns1"
    name                         = "new-app-events"
    batched_operations_enabled   = false
    express_enabled              = false
    requires_duplicate_detection = false
    support_ordering             = false
  }
}
```

#### Step 2: Add subscription configuration

```hcl
servicebus_subscriptions = {
  # ... existing subscriptions ...
  
  "nap" = {
    topic_key                                 = "na"
    name                                      = "new-app-processor"
    dead_lettering_on_message_expiration      = false
    dead_lettering_on_filter_evaluation_error = false
    batched_operations_enabled                = false
    requires_session                          = false
    max_delivery_count                        = 3
    client_id                                 = ""
  }
}
```

No changes to `.tf` files required - existing modules handle the creation!

---

## Best Practices

### 1. Naming Consistency

* Always use the naming pattern defined in `locals.tf`
* Never hardcode resource names in modules
* Use environment and location abbreviations consistently

### 2. Module Versioning

* Pin module versions using `version = "x.y.z"`
* Avoid using `version = "~> x.y"` in production
* Review module changes before upgrading versions

### 3. Security

* **Never disable** RBAC authorization on Key Vaults
* Use **Azure AD authentication only** for SQL Servers (no SQL auth)
* Enable **Entra ID authentication** for Redis (disable access keys)
* Use **private endpoints** for all PaaS services
* Implement **network security** with IP allowlists and subnet restrictions

### 4. Tags

* Apply consistent tags across all resources
* Include at minimum:
  * `Owner`
  * `Environment`
  * `CreatedBy`
  * `Location`
  * `CostCenter`
  * `Criticality`
  * `ApplicationName`
  * `TFManaged`
  * `OpsTeam`

### 5. Identity Management

* Use **User Assigned Managed Identities** over service principals
* Implement **Federated Credentials** for GitHub Actions (no secrets!)
* Use **Workload Identity** for AKS pods
* Apply **least privilege** principle with role assignments

### 6. Configuration Management

* Keep environment-specific configuration in `.auto.tfvars` files
* Use variables for all configurable values
* Document variable defaults in `variables.tf`

### 7. State Management

* Use remote state backend (likely Spacelift)
* Never commit `.tfstate` files
* Use state locking

### 8. Private Endpoints

* Always create private endpoints for:
  * Key Vault
  * Storage Accounts
  * SQL Server
  * Redis Cache
  * Service Bus
  * App Configuration
  * AKS (private cluster when required)

### 9. DNS Configuration

* All private DNS zones are centrally managed in `prd-eus2-identity-dns-rg-001`
* Reference DNS zone IDs from locals, don't hardcode

### 10. Maintenance Windows

* Configure maintenance windows for AKS clusters
* Schedule during low-traffic periods
* Use CST timezone offset (`-06:00`)

---

## Troubleshooting

### Common Issues

#### Issue 1: Key Vault Access Denied During Deployment

**Symptom**: `Error: authorization failed` when creating Key Vault secrets

**Solution**: Ensure the deployment identity has `Key Vault Secrets Officer` role:

```hcl
module "assign_deployment_uami_key_vault_secrets_officer" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  scope           = module.key_vault.id
  role_definition = "Key Vault Secrets Officer"
  principal_id    = data.azurerm_client_config.current.object_id
}
```

Add `depends_on` to secret creation:

```hcl
module "kv_secrets" {
  # ... configuration ...
  depends_on = [module.assign_deployment_uami_key_vault_secrets_officer]
}
```

#### Issue 2: Private Endpoint DNS Resolution Failing

**Symptom**: Cannot connect to resource via private endpoint

**Solution**:

1. Verify private DNS zone exists in central DNS resource group
2. Confirm DNS zone ID in `locals.tf` is correct
3. Check VNet is linked to private DNS zone
4. Verify private endpoint is in the correct subnet

#### Issue 3: Storage Account Name Conflict

**Symptom**: `Error: storage account name already exists`

**Solution**: Storage account names must be globally unique and lowercase. Check:

1. Naming logic in `locals.tf` generates unique names
2. No special characters (only alphanumeric)
3. Maximum 24 characters
4. Storage account key in variable matches naming logic

#### Issue 4: AKS Cluster Upgrade Failure

**Symptom**: AKS upgrade fails during maintenance window

**Solution**:

1. Check node pool SKU availability in target region
2. Verify Kubernetes version is supported
3. Review maintenance window configuration
4. Check for pod disruption budgets blocking drain

#### Issue 5: Service Bus Permission Denied

**Symptom**: Application cannot send/receive messages

**Solution**: Verify UAMI has correct role assignment flags:

```hcl
user_assigned_identitys = {
  "app-name" = {
    service_bus_sender = true      # For sending messages
    service_bus_reciever = true    # For receiving messages
  }
}
```

#### Issue 6: Module Version Not Found

**Symptom**: `Error: module version not found in registry`

**Solution**:

1. Verify you have access to Spacelift registry
2. Check module version exists
3. Ensure provider authentication is configured
4. Try `terraform init -upgrade`

#### Issue 7: For\_Each Errors

**Symptom**: `Error: Invalid for_each argument`

**Solution**:

1. Ensure the variable is a map, not a list
2. Verify all map keys are strings
3. Check for null values in map
4. Validate variable structure matches expected schema

---

## Deployment Workflow

### Initial Setup

1. **Clone Repository**

```bash
git clone <repository-url>
cd az-adc-lz-tf
```

2. **Select Layer and Environment**

```bash
cd app  # or networking, shared_services
```

3. **Initialize Terraform**

```bash
terraform init
```

4. **Select Workspace** (if using workspaces)

```bash
terraform workspace select dev-eastus2
# or
terraform workspace new dev-eastus2
```

### Making Changes

1. **Update Configuration**
   * Modify `.auto.tfvars` file for environment-specific changes
   * Update `.tf` files for structural changes

2. **Plan Changes**

```bash
terraform plan -var-file="config/dev.eastus2.auto.tfvars"
```

3. **Review Plan**
   * Verify expected resources will be created/modified/destroyed
   * Check for unintended changes

4. **Apply Changes**

```bash
terraform apply -var-file="config/dev.eastus2.auto.tfvars"
```

### Layer Deployment Order

When deploying a new environment, follow this order:

1. **Networking Layer** - Foundation network infrastructure
2. **Shared Services Layer** - Cross-application resources
3. **Application Layer** - Application-specific resources

### Spacelift Integration

If using Spacelift for automation:

* Stacks are likely configured per layer per environment
* Configuration files (`.auto.tfvars`) trigger stack runs
* Review plans in Spacelift UI before confirming
* Use stack dependencies to enforce deployment order

---

## Additional Resources

### Private DNS Zones (Centrally Managed)

All private DNS zones are managed in subscription `b83668dc-1957-4664-b687-7bd4ae39d41a`:

| Service           | DNS Zone                              |
| ----------------- | ------------------------------------- |
| AKS               | `privatelink.eastus2.azmk8s.io`       |
| Storage Blob      | `privatelink.blob.core.windows.net`   |
| SQL Database      | `privatelink.database.windows.net`    |
| Key Vault         | `privatelink.vaultcore.azure.net`     |
| Function App      | `privatelink.azurewebsites.net`       |
| Service Bus       | `privatelink.servicebus.windows.net`  |
| Redis             | `privatelink.redis.cache.windows.net` |
| App Configuration | `privatelink.azconfig.io`             |

### Pre-deployed Resources

Some resources are pre-deployed and referenced via data sources:

* **AKS UAMI**: `{env}-{loc}-{app}-aks-uami-001`
* **Application Gateway UAMI**: `{env}-{loc}-apim-hub-webagw-uami-001`
* **App Insights**: `{env}-{loc}-{app}-appinsights-001`
* **Operations Log Analytics**: `prdeus2opslog001` (production subscription)

### Contact Information

For questions or issues:

* **Owner**: Aaron Alex
* **Ops Team**: Data Operations
* **Support**: Jon Cormack

---

## Appendix: Variable Reference

### Common Variables

```hcl
# Required in all layers
variable "location" {
  description = "Azure region"
  type        = string
}

variable "location_abbreviation" {
  description = "Region abbreviation (eus2, cus)"
  type        = string
}

variable "environment_abbreviation" {
  description = "Environment abbreviation (dev, qa, uat, prd, sbx)"
  type        = string
}

variable "tags" {
  description = "Resource tags"
  type        = map(string)
}
```

### Security Variables

```hcl
variable "ip_rules" {
  description = "Allowed IP addresses for PaaS services"
  type        = list(string)
}

variable "virtual_network_rule_ids" {
  description = "Subnet IDs allowed to access PaaS services"
  type        = list(string)
}
```

---

## GitHub Copilot Instructions

### Overview

To enhance your development experience with GitHub Copilot in this repository, you can create a `.github/copilot-instructions.md` file that provides context-specific guidance to Copilot about this infrastructure codebase.

### Creating the Instructions File

#### Step 1: Create the Directory Structure

```bash
mkdir -p .github
```

#### Step 2: Create the Instructions File

Create `.github/copilot-instructions.md` with the following content:

```markdown
# GitHub Copilot Instructions for Azure ADC Landing Zone

## Repository Context

This is a Terraform Infrastructure as Code (IaC) repository for Azure Landing Zone components for the Application Development Center (ADC) environment. It uses Spacelift modules from the private registry `us.spacelift.io/claconnect`.

## Architecture Pattern

### Three-Layer Architecture
1. **Networking Layer** (`networking/`) - VNets, subnets, network security
2. **Shared Services Layer** (`shared_services/`) - ACR, shared Key Vaults, storage
3. **Application Layer** (`app/`) - AKS, SQL, Redis, Service Bus, UAMIs, App Registrations

## Naming Conventions

Always follow this pattern:
```

\{environment}-\{location}-\{application}-\{resource\_type}-\{suffix}

```

### Abbreviations
- Environments: `dev`, `qa`, `uat`, `prd`, `sbx`
- Locations: `eus2` (East US 2), `cus` (Central US)
- Application: `adc`

### Examples
- Resource Group: `dev-eus2-adc-app-rg-001`
- AKS Cluster: `dev-eus2-adc-aks-001`
- Storage Account: `deveus2adcinst001` (lowercase, no hyphens)

**IMPORTANT**: All resource names are computed in `locals.tf`. Never hardcode names in module calls.

## Module Usage Guidelines

### Module Source Pattern
```

```hcl
module "example" {
  source  = "us.spacelift.io/claconnect/<module_name>/<provider>"
  version = "x.y.z"
  # ... configuration
}
```

### Always Pin Versions

* Use exact versions: `version = "1.0.5"`
* Never use: `version = "~> 1.0"` in production code

## Security Requirements

### Authentication & Authorization

* **SQL Servers**: Always use Azure AD authentication only (`administrator_login = null`, `azuread_administrator_azread_authentication_only = true`)
* **Key Vaults**: Always enable RBAC authorization (`rbac_authorization_enabled = true`)
* **Redis**: Always use Entra ID authentication (`access_keys_authentication_enabled = false`)
* **Service Bus**: Disable local auth (`local_auth_enabled = false`)

### Network Security

* Always create private endpoints for PaaS services
* Use `network_default_action = "Deny"` with explicit IP allowlists
* Reference centralized private DNS zones from `locals.tf`

### Identity Management

* Prefer User Assigned Managed Identities over service principals
* Use Federated Credentials for GitHub Actions (no secrets!)
* Implement Workload Identity for AKS pods
* Apply least privilege with permission flags

## Code Patterns

### User Assigned Managed Identities (UAMIs)

When suggesting UAMI additions, use permission flags in tfvars:

```hcl
user_assigned_identitys = {
  "app-name" = {
    federated_credentials = {
      github = {
        repo = "repo-name"
      }
      workload_identity = {
        namespace           = "namespace"
        service_account_name = "sa-name"
      }
    }
    # Permission flags (set to true as needed)
    key_vault_secrets_user                = false
    key_vault_secrets_user_02             = false
    key_vault_secrets_officer             = false
    app_config_reader                     = false
    service_bus_sender                    = false
    service_bus_reciever                  = false
    input_storage_blob_data_contributor   = false
    input_storage_blob_data_reader        = false
    documents_storage_blob_data_contributor = false
    documents_storage_blob_data_reader    = false
    app_insights_publisher                = false
    aks_cluster_admin                     = false
  }
}
```

### Role Assignments Pattern

Role assignments use for\_each with lookup pattern:

```hcl
module "assign_uamis_[permission]" {
  source  = "us.spacelift.io/claconnect/role_assignment/azurerm"
  version = "1.1.1"

  for_each = { for k, v in var.user_assigned_identitys : k => v if lookup(v, "permission_flag", false) == true }

  scope           = module.resource.id
  role_definition = "Azure Role Name"
  principal_id    = module.user_assigned_identity[each.key].principal_id
}
```

### Private Endpoint Pattern

Always create private endpoints with DNS integration:

```hcl
module "resource_private_endpoint" {
  source  = "us.spacelift.io/claconnect/private_endpoint/azurerm"
  version = "1.0.4"

  name                  = "${local.resource_name}-pe"
  location              = var.location
  resource_group_name   = module.resource_group.name
  subnet_id             = data.azurerm_subnet.pe_subnet.id
  tags                  = var.tags
  private_dns_zone_name = local.service_private_dns_zone_name
  private_dns_zone_ids  = [local.service_private_dns_zone_id]
  
  private_service_connection = {
    name                           = "${local.resource_name}-pe"
    private_connection_resource_id = module.resource.id
    subresource_names              = ["subresource"]
    is_manual_connection           = false
  }
}
```

### Storage Account Pattern

Storage accounts use for\_each over a variable map:

```hcl
# In locals.tf
storage_account_names = { 
  for k, v in var.storage_accounts : 
  k => "${var.environment_abbreviation}${var.location_abbreviation}${var.application_abbreviation}${k}st001" 
}

# In storage.tf
module "storage_account" {
  source   = "us.spacelift.io/claconnect/storage_account/azurerm"
  version  = "1.0.10"
  for_each = var.storage_accounts

  name                          = local.storage_account_names[each.key]
  # ... configuration
}
```

## Configuration Best Practices

### Variable Configuration Location

* Environment-specific values go in `config/*.auto.tfvars`
* Variable declarations go in `variables.tf`
* Computed values go in `locals.tf`
* Never hardcode values in `.tf` files

### Tags

Always include these tags:

```hcl
tags = {
  Owner           = "Owner Name"
  Environment     = "Development|QA|UAT|Production"
  CreatedBy       = "Terraform"
  Location        = "EastUS2|CentralUS"
  CostCenter      = "Cost Center"
  Criticality     = "Low|Medium|High|Critical"
  ApplicationName = "ADC"
  TFManaged       = "True"
  OpsTeam         = "Team Name"
  CreatedDate     = "MM-DD-YYYY"
}
```

### Service Bus Pattern

Service Bus uses three-tier configuration:

1. **Namespace** - One per environment
2. **Topics** - Multiple per namespace
3. **Subscriptions** - Multiple per topic

```hcl
servicebus_topics = {
  "key" = {
    namespace_key = "ns1"
    name          = "topic-name"
    # ... config
  }
}

servicebus_subscriptions = {
  "key" = {
    topic_key = "topic-key-reference"
    name      = "subscription-name"
    # ... config
  }
}
```

## File Organization

### Application Layer Files

* `main.tf` - Resource group only
* `provider.tf` - Provider configuration
* `variables.tf` - All variable declarations (1100+ lines)
* `locals.tf` - Naming logic and computed values
* `data.tf` - Data source queries
* `aks_cluster.tf` - AKS resources
* `database.tf` - SQL resources
* `redis.tf` - Redis cache
* `kv.tf`, `kv_02.tf` - Key Vaults
* `storage.tf` - Storage accounts
* `servicebus.tf` - Service Bus
* `managed_identity.tf` - UAMIs
* `app_reg.tf` - App registrations
* `roles.tf` - Role assignments
* `app_config.tf` - App Configuration
* `output.tf` - Outputs

## Common Patterns to Suggest

### Adding a New Resource Type

1. Add naming logic to `locals.tf`
2. Add variable declarations to `variables.tf` with defaults
3. Create module in appropriate `.tf` file
4. Create private endpoint (if applicable)
5. Add values to `config/*.auto.tfvars`

### Adding a New UAMI

Only modify tfvars - no `.tf` changes needed:

```hcl
user_assigned_identitys = {
  "new-app" = {
    federated_credentials = { ... }
    key_vault_secrets_user = true
    service_bus_sender     = true
  }
}
```

### Adding Storage Permissions to Existing UAMI

1. Add role assignment module in `roles.tf`
2. Add permission flag to UAMI in tfvars

## What NOT to Suggest

* ❌ Hardcoded resource names in modules
* ❌ SQL authentication with username/password
* ❌ Access keys for Redis
* ❌ Service Bus with local auth enabled
* ❌ Resources without tags
* ❌ PaaS services without private endpoints
* ❌ Unpinned module versions
* ❌ Creating new AKS clusters (singleton pattern)

## DNS Zones Reference

All private DNS zones are centrally managed in subscription `b83668dc-1957-4664-b687-7bd4ae39d41a`:

* AKS: `privatelink.eastus2.azmk8s.io`
* Storage Blob: `privatelink.blob.core.windows.net`
* SQL: `privatelink.database.windows.net`
* Key Vault: `privatelink.vaultcore.azure.net`
* Service Bus: `privatelink.servicebus.windows.net`
* Redis: `privatelink.redis.cache.windows.net`
* App Configuration: `privatelink.azconfig.io`

## Deployment Context

* Using Spacelift for CI/CD
* Deploy order: Networking → Shared Services → Application
* Each layer/environment likely has its own Spacelift stack
* Changes to `.auto.tfvars` trigger deployments

## When Suggesting Code

1. Always follow existing patterns in the repository
2. Use the same module versions as similar resources
3. Include all required security configurations
4. Add both the resource and its private endpoint
5. Reference locals for names, not hardcode
6. Include depends\_on when necessary (e.g., role assignments)
7. Suggest tfvars changes when possible instead of .tf changes
8. Include tags in all resources

## Contact Information

* **Owner**: Aaron Alex
* **Ops Team**: Data Operations
* **Support**: Jon Cormack

```

#### Step 3: Commit the File

```

```bash
git add .github/copilot-instructions.md
git commit -m "Add GitHub Copilot instructions for repository context"
git push
```

### How It Works

Once this file is in place:

1. **Automatic Context**: GitHub Copilot will automatically read and use these instructions when providing suggestions in this repository
2. **Consistent Patterns**: Copilot will follow the naming conventions, security requirements, and code patterns defined
3. **Better Suggestions**: Copilot will understand the three-layer architecture and suggest appropriate module usage
4. **Security Compliance**: Copilot will suggest secure configurations by default (RBAC, private endpoints, etc.)

### Benefits

* **Faster Development**: Copilot understands your repository structure and patterns
* **Consistency**: All suggestions follow established conventions
* **Security**: Security-first suggestions aligned with best practices
* **Reduced Errors**: Copilot avoids anti-patterns specific to this codebase
* **Team Alignment**: New team members get consistent AI assistance

### Testing the Instructions

After adding the file, test it by:

1. Opening a `.tf` file in VS Code
2. Starting to type a new resource (e.g., "module "new\_redis")
3. Observing that Copilot suggestions follow the patterns defined in the instructions
4. Asking Copilot Chat questions like "How do I add a new UAMI with storage permissions?"

### Updating Instructions

As the repository evolves, update `.github/copilot-instructions.md` to:

* Add new patterns and conventions
* Document new resource types
* Update security requirements
* Add lessons learned from common mistakes

---

**Document Version**: 1.0\
**Last Updated**: January 22, 2026\
**Terraform Version**: >= 1.0\
**Module Registry**: us.spacelift.io/claconnect