Terraform

Question

我有一个 Terraform 脚本可以创建 Azure Key Vault，导入我的 SSL 证书（带密码的 3DES .pfx 文件），并创建一个带有 HTTP 侦听器的应用程序网关。我正在尝试将其更改为使用我来自 KeyVault 的 SSL 证书的 HTTPS 侦听器。

我已经在 Azure 门户中手动完成了这个过程，并且我已经使用 PowerShell 完成了这个过程。不幸的是，我没有找到 Terraform 的文档清楚地说明应该如何实现。

以下是我的应用程序网关和证书资源的相关片段：

resource "azurerm_application_gateway" "appgw" {
  name                = "my-appgw"
  location            = "australiaeast"
  resource_group_name = "my-rg"
  
  http_listener {
    protocol                       = "https"
    ssl_certificate_name           = "appgw-listener-cert"
    ...
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.appgw_uaid.id]
  }

  ssl_certificate {
    key_vault_secret_id = azurerm_key_vault_certificate.ssl_cert.secret_id
    name                = "appgw-listener-cert"
  }

  ...
}

resource "azurerm_key_vault" "kv" {
  name                       = "my-kv"
  location                   = "australiaeast"
  resource_group_name        = "my-rg"
  ...
  access_policy {
    object_id    = data.azurerm_client_config.current.object_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    certificate_permissions = [
      "Create",
      "Delete",
      "DeleteIssuers",
      "Get",
      "GetIssuers",
      "Import",
      "List",
      "ListIssuers",
      "ManageContacts",
      "ManageIssuers",
      "Purge",
      "SetIssuers",
      "Update"
    ]

    key_permissions = [
      "Backup",
      "Create",
      "Decrypt",
      "Delete",
      "Encrypt",
      "Get",
      "Import",
      "List",
      "Purge",
      "Recover",
      "Restore",
      "Sign",
      "UnwrapKey",
      "Update",
      "Verify",
      "WrapKey"
    ]

    secret_permissions = [
      "Backup",
      "Delete",
      "Get",
      "List",
      "Purge",
      "Restore",
      "Restore",
      "Set"
    ]
  }

  access_policy {
    object_id    = azurerm_user_assigned_identity.uaid_appgw.principal_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    secret_permissions = [
      "Get"
    ]
  }
}

resource "azurerm_key_vault_certificate" "ssl_cert" {
  name         = "my-ssl-cert"
  key_vault_id = azurerm_key_vault.kv.id

  certificate {
    # These are stored as sensitive variables in Terraform Cloud
    # ssl_cert_b64 value was retrieved by: $ cat my-ssl-cert.pfx | base64 > o.txt
    contents = var.ssl_cert_b64
    password = var.ssl_cert_passwd
  }

  certificate_policy {
    issuer_parameters {
      name = "Unknown"
    }

    key_properties {
      exportable = false
      key_size   = 2048
      key_type   = "RSA"
      reuse_key  = false
    }

    secret_properties {
      content_type = "application/x-pkcs12"
    }
  }
}

这是我在 Terraform Cloud 中遇到的（经过清理的）错误：

Error: waiting for create/update of Application Gateway: (Name "my-appgw" / Resource Group "my-rg"): Code="ApplicationGatewayKeyVaultSecretException" Message="Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/1324/resourceGroups/my-rg/providers/Microsoft.Network/applicationGateways/my-appgw'. See details below:" Details=[{"code":"ApplicationGatewaySslCertificateDoesNotHavePrivateKey","message":"Certificate /subscriptions/1324/resourceGroups/my-rg/providers/Microsoft.Network/applicationGateways/my-appgw/sslCertificates/appgw-listener-cert does not have Private Key."}]

我从 Key Vault 下载了证书，它似乎是有效的，没有损坏或损坏。我不明白为什么错误说它没有私钥。

有人可以指出我遗漏了什么或做错了什么吗？

Answer 1

问题是密钥保管库中没有为应用程序网关定义任何访问策略，它无法为其获取证书。

因此，为了解决这个问题，您必须为应用程序网关使用的托管标识添加访问策略。因此，在创建托管身份之后和在应用程序网关中使用之前，您必须使用如下内容：

provider "azurerm" {
    features{}
}
data "azurerm_client_config" "current" {}

resource "azurerm_user_assigned_identity" "base" {
  resource_group_name = "yourresourcegroup"
  location            = "resourcegrouplocation"
  name                = "mi-appgw-keyvault"
}

data "azurerm_key_vault" "example"{
    name = "testansumankeyvault-01"
    resource_group_name = "yourresourcegroup"
} 
resource "azurerm_key_vault_access_policy" "example" {
  key_vault_id = data.azurerm_key_vault.example.id
  tenant_id    = data.azurerm_client_config.current.tenant_id
  object_id    = azurerm_user_assigned_identity.base.principal_id

  key_permissions = [
    "Get",
  ]

  certificate_permissions = [
      "Get",
  ]

  secret_permissions = [
    "Get",
  ]
}

所以，只有在完成上述操作后，您才可以根据需要使用类似于以下的内容：

data "azurerm_user_assigned_identity" "example" {
  name                = "mi-appgw-keyvault"
  resource_group_name = "yourresourcegroup"
}
data "azurerm_key_vault" "example"{
    name = "testansumankeyvault-01"
    resource_group_name = "yourresourcegroup"
} 
resource "azurerm_application_gateway" "appgw" {
  name                = "my-appgw"
  location            = "australiaeast"
  resource_group_name = "my-rg"
  
  http_listener {
    protocol                       = "https"
    ssl_certificate_name           = "appgw-listener-cert"
    ...
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [data.azurerm_user_assigned_identity.example.id]
  }

  ssl_certificate {
    key_vault_secret_id = azurerm_key_vault_certificate.ssl_cert.secret_id
    name                = "appgw-listener-cert"
  }

  ...
}
    
data "azurerm_key_vault_certificate" "example" {
  name         = "secret-sauce"
  key_vault_id = data.azurerm_key_vault.example.id
}

注：

我已经使用 exisitng keyvault 设置用于测试的 keyvault 访问策略以及 keyvault 中的现有证书。如果您正在创建新的，请使用 2 个部署：

部署 Keyvault，managed_identity，首先为 keyvault 访问策略和证书。
然后将数据源用于 keyvault、托管身份和证书，然后使用从 keyvault 引用的 ssl 证书部署应用程序网关。

Answer 2

我在我的环境中测试了 2 个场景：

场景一：在Keyvault中生成新的证书，上传到应用网关ssl证书中。

provider "azurerm" {
    features{}
}
data "azurerm_client_config" "current" {}

data "azurerm_resource_group" "example"{
    name = "ansumantest"
}

resource "azurerm_user_assigned_identity" "base" {
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  name                = "mi-appgw-keyvault"
}


resource "azurerm_key_vault" "kv" {
  name                       = "ansumankeyvault01"
  location                   = data.azurerm_resource_group.example.location
  resource_group_name        = data.azurerm_resource_group.example.name
  tenant_id = data.azurerm_client_config.current.tenant_id
  sku_name = "standard"
  access_policy {
    object_id    = data.azurerm_client_config.current.object_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    certificate_permissions = [
      "Create",
      "Delete",
      "DeleteIssuers",
      "Get",
      "GetIssuers",
      "Import",
      "List",
      "ListIssuers",
      "ManageContacts",
      "ManageIssuers",
      "Purge",
      "SetIssuers",
      "Update"
    ]

    key_permissions = [
      "Backup",
      "Create",
      "Decrypt",
      "Delete",
      "Encrypt",
      "Get",
      "Import",
      "List",
      "Purge",
      "Recover",
      "Restore",
      "Sign",
      "UnwrapKey",
      "Update",
      "Verify",
      "WrapKey"
    ]

    secret_permissions = [
      "Backup",
      "Delete",
      "Get",
      "List",
      "Purge",
      "Restore",
      "Restore",
      "Set"
    ]
  }

  access_policy {
    object_id    = azurerm_user_assigned_identity.base.principal_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    secret_permissions = [
      "Get"
    ]
  }
}

output "secret_identifier" {
  value = azurerm_key_vault_certificate.example.secret_id
}

resource "azurerm_key_vault_certificate" "example" {
  name         = "generated-cert"
  key_vault_id = azurerm_key_vault.kv.id

  certificate_policy {
    issuer_parameters {
      name = "Self"
    }

    key_properties {
      exportable = true
      key_size   = 2048
      key_type   = "RSA"
      reuse_key  = true
    }

    lifetime_action {
      action {
        action_type = "AutoRenew"
      }

      trigger {
        days_before_expiry = 30
      }
    }

    secret_properties {
      content_type = "application/x-pkcs12"
    }

    x509_certificate_properties {
      # Server Authentication = 1.3.6.1.5.5.7.3.1
      # Client Authentication = 1.3.6.1.5.5.7.3.2
      extended_key_usage = ["1.3.6.1.5.5.7.3.1"]

      key_usage = [
        "cRLSign",
        "dataEncipherment",
        "digitalSignature",
        "keyAgreement",
        "keyCertSign",
        "keyEncipherment",
      ]

      subject_alternative_names {
        dns_names = ["internal.contoso.com", "domain.hello.world"]
      }

      subject            = "CN=hello-world"
      validity_in_months = 12
    }
  }
}

resource "azurerm_virtual_network" "example" {
  name                = "example-network"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  address_space       = ["10.254.0.0/16"]
}

resource "azurerm_subnet" "frontend" {
  name                 = "frontend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.0.0/24"]
}

resource "azurerm_subnet" "backend" {
  name                 = "backend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.2.0/24"]
}

resource "azurerm_public_ip" "example" {
  name                = "example-pip"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  allocation_method   = "Static"
  sku = "standard"
}

#&nbsp;since these variables are re-used - a locals block makes this more maintainable
locals {
  backend_address_pool_name      = "${azurerm_virtual_network.example.name}-beap"
  frontend_port_name             = "${azurerm_virtual_network.example.name}-feport"
  frontend_ip_configuration_name = "${azurerm_virtual_network.example.name}-feip"
  http_setting_name              = "${azurerm_virtual_network.example.name}-be-htst"
  listener_name                  = "${azurerm_virtual_network.example.name}-httplstn"
  request_routing_rule_name      = "${azurerm_virtual_network.example.name}-rqrt"
  redirect_configuration_name    = "${azurerm_virtual_network.example.name}-rdrcfg"

}

resource "null_resource" "previous" {}

resource "time_sleep" "wait_240_seconds" {
  depends_on = [azurerm_key_vault.kv]

  create_duration = "240s"
}

resource "azurerm_application_gateway" "network" {
  name                = "example-appgateway"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location

  sku {
    name     = "Standard_v2"
    tier     = "Standard_v2"
    capacity = 2
  }

  gateway_ip_configuration {
    name      = "my-gateway-ip-configuration"
    subnet_id = azurerm_subnet.frontend.id
  }

  frontend_port {
    name = local.frontend_port_name
    port = 443
  }

  frontend_ip_configuration {
    name                 = local.frontend_ip_configuration_name
    public_ip_address_id = azurerm_public_ip.example.id
  }

  backend_address_pool {
    name = local.backend_address_pool_name
  }

  backend_http_settings {
    name                  = local.http_setting_name
    cookie_based_affinity = "Disabled"
    path                  = "/path1/"
    port                  = 443
    protocol              = "Https"
    request_timeout       = 60
  }

  http_listener {
    name                           = local.listener_name
    frontend_ip_configuration_name = local.frontend_ip_configuration_name
    frontend_port_name             = local.frontend_port_name
    protocol                       = "Https"
    ssl_certificate_name = "app_listener"
  }

  identity {
    type = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.base.id]
  }

  ssl_certificate {
    name = "app_listener"
    key_vault_secret_id = azurerm_key_vault_certificate.example.secret_id
  }

  request_routing_rule {
    name                       = local.request_routing_rule_name
    rule_type                  = "Basic"
    http_listener_name         = local.listener_name
    backend_address_pool_name  = local.backend_address_pool_name
    backend_http_settings_name = local.http_setting_name
  }
  depends_on = [time_sleep.wait_240_seconds]
}

输出：

场景 2：使用我从本地计算机导入到 keyvault 的一个证书，并在应用程序网关中使用它。

provider "azurerm" {
    features{}
}
data "azurerm_client_config" "current" {}

data "azurerm_resource_group" "example"{
    name = "ansumantest"
}

resource "azurerm_user_assigned_identity" "base" {
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  name                = "mi-appgw-keyvault"
}


resource "azurerm_key_vault" "kv" {
  name                       = "ansumankeyvault01"
  location                   = data.azurerm_resource_group.example.location
  resource_group_name        = data.azurerm_resource_group.example.name
  tenant_id = data.azurerm_client_config.current.tenant_id
  sku_name = "standard"
  access_policy {
    object_id    = data.azurerm_client_config.current.object_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    certificate_permissions = [
      "Create",
      "Delete",
      "DeleteIssuers",
      "Get",
      "GetIssuers",
      "Import",
      "List",
      "ListIssuers",
      "ManageContacts",
      "ManageIssuers",
      "Purge",
      "SetIssuers",
      "Update"
    ]

    key_permissions = [
      "Backup",
      "Create",
      "Decrypt",
      "Delete",
      "Encrypt",
      "Get",
      "Import",
      "List",
      "Purge",
      "Recover",
      "Restore",
      "Sign",
      "UnwrapKey",
      "Update",
      "Verify",
      "WrapKey"
    ]

    secret_permissions = [
      "Backup",
      "Delete",
      "Get",
      "List",
      "Purge",
      "Restore",
      "Restore",
      "Set"
    ]
  }

  access_policy {
    object_id    = azurerm_user_assigned_identity.base.principal_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    secret_permissions = [
      "Get"
    ]
  }
}

output "secret_identifier" {
  value = azurerm_key_vault_certificate.example.secret_id
}

resource "azurerm_key_vault_certificate" "example" {
  name         = "imported-cert"
  key_vault_id = azurerm_key_vault.kv.id

  certificate {
    contents = filebase64("C:/appgwlistner.pfx")
    password = "password"
  }

  certificate_policy {
    issuer_parameters {
      name = "Self"
    }

    key_properties {
      exportable = true
      key_size   = 2048
      key_type   = "RSA"
      reuse_key  = false
    }

    secret_properties {
      content_type = "application/x-pkcs12"
    }
  }
}

resource "azurerm_virtual_network" "example" {
  name                = "example-network"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  address_space       = ["10.254.0.0/16"]
}

resource "azurerm_subnet" "frontend" {
  name                 = "frontend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.0.0/24"]
}

resource "azurerm_subnet" "backend" {
  name                 = "backend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.2.0/24"]
}

resource "azurerm_public_ip" "example" {
  name                = "example-pip"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  allocation_method   = "Static"
  sku = "standard"
}

#&nbsp;since these variables are re-used - a locals block makes this more maintainable
locals {
  backend_address_pool_name      = "${azurerm_virtual_network.example.name}-beap"
  frontend_port_name             = "${azurerm_virtual_network.example.name}-feport"
  frontend_ip_configuration_name = "${azurerm_virtual_network.example.name}-feip"
  http_setting_name              = "${azurerm_virtual_network.example.name}-be-htst"
  listener_name                  = "${azurerm_virtual_network.example.name}-httplstn"
  request_routing_rule_name      = "${azurerm_virtual_network.example.name}-rqrt"
  redirect_configuration_name    = "${azurerm_virtual_network.example.name}-rdrcfg"

}

resource "null_resource" "previous" {}

resource "time_sleep" "wait_240_seconds" {
  depends_on = [azurerm_key_vault.kv]

  create_duration = "240s"
}

resource "azurerm_application_gateway" "network" {
  name                = "example-appgateway"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location

  sku {
    name     = "Standard_v2"
    tier     = "Standard_v2"
    capacity = 2
  }

  gateway_ip_configuration {
    name      = "my-gateway-ip-configuration"
    subnet_id = azurerm_subnet.frontend.id
  }

  frontend_port {
    name = local.frontend_port_name
    port = 443
  }

  frontend_ip_configuration {
    name                 = local.frontend_ip_configuration_name
    public_ip_address_id = azurerm_public_ip.example.id
  }

  backend_address_pool {
    name = local.backend_address_pool_name
  }

  backend_http_settings {
    name                  = local.http_setting_name
    cookie_based_affinity = "Disabled"
    path                  = "/path1/"
    port                  = 443
    protocol              = "Https"
    request_timeout       = 60
  }

  http_listener {
    name                           = local.listener_name
    frontend_ip_configuration_name = local.frontend_ip_configuration_name
    frontend_port_name             = local.frontend_port_name
    protocol                       = "Https"
    ssl_certificate_name = "app_listener"
  }

  identity {
    type = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.base.id]
  }

  ssl_certificate {
    name = "app_listener"
    key_vault_secret_id = azurerm_key_vault_certificate.example.secret_id
  }

  request_routing_rule {
    name                       = local.request_routing_rule_name
    rule_type                  = "Basic"
    http_listener_name         = local.listener_name
    backend_address_pool_name  = local.backend_address_pool_name
    backend_http_settings_name = local.http_setting_name
  }
  depends_on = [time_sleep.wait_240_seconds]
}

输出：

注：

请确保拥有带私钥的 pfx 证书。使用安全证书导出pfx证书时，请确保如下所示选择以下属性，然后输入密码并导出。

Terraform - 如何将存储在 Azure KeyVault 中的 SSL 证书附加到应用程序网关

Terraform - How to attach SSL certificate stored in Azure KeyVault to an Application Gateway

ssl-certificate

azure-keyvault

azure-application-gateway