Terraform：使用流量管理器端点部署 Kubernetes 服务

Question

我需要使用 terraform 在流量管理器中添加 Kubernetes 服务的解决方案，为了做到这一点，我需要为每个集群提供一个 public IP 地址，但似乎 IP 是在不同的环境下创建的部署后订阅。

尝试使用 azurerm_traffic_manager_endpoint 不同的类型，如 azureEndpoints 和 nestedEndpoints，但似乎脚本失败并出现下面列出的相同错误。

下面是我要部署的脚本，我将分享错误：

错误：

creating/updating nestedEndpoints Endpoint "vmap-tmep" (Traffic Manager Profile "vmap-tm" / Resource Group "RG-TEST-TEST"): trafficmanager.EndpointsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="The 'resourceTargetId' property of endpoint 'vmap-tmep' is invalid or missing. The property must be specified only for the following endpoint types: AzureEndpoints, NestedEndpoints. You must have read access to the resource to which it refers."


# Traffic Manager Profile Resource
resource "azurerm_traffic_manager_profile" "tmp" {
  name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-tm")
  resource_group_name = azurerm_resource_group.rg.name
  
  traffic_routing_method = "Weighted"

  dns_config {
    relative_name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-tm-dns-test")
    ttl           = 100
  }

  monitor_config {
    protocol                     = "http"
    port                         = 80
    path                         = "/"
    interval_in_seconds          = 30
    timeout_in_seconds           = 9
    tolerated_number_of_failures = 3
  }
}

# Traffic Manager Endpoint Resource
resource "azurerm_traffic_manager_endpoint" "tmep" {
  name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-tmep")
  resource_group_name = azurerm_resource_group.rg.name
  profile_name        = azurerm_traffic_manager_profile.tmp.name
  type                = "nestedEndpoints"
  weight              = 1000
  target_resource_id = azurerm_kubernetes_cluster.k8s1.id
}



################ K8S nodes pool location 1 ################

resource "azurerm_kubernetes_cluster" "k8s1" {
  name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-k8s")
  location            = var.location
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = "exampleaks1"

  service_principal {
    client_id = "bsdfsdfs3b"
    client_secret = "353sdfsdfsdfsdfsd9"
  }
    role_based_access_control {
      azure_active_directory {
        managed           = true
        admin_group_object_ids = [var.group_object_id]
        tenant_id         = var.tenant_id
        azure_rbac_enabled = true
      }
        enabled = true
    
    }   

  linux_profile {
    admin_username = var.adminusername

    ssh_key {
      key_data = "${file("${var.ssh_public_key}")}"
    }
  }
auto_scaler_profile {
           new_pod_scale_up_delay           = "5s" 
           scale_down_delay_after_delete    = "10s" 
           skip_nodes_with_local_storage    = false
        }
addon_profile {
            azure_policy{
        enabled                    = true
      }
}


 default_node_pool {
          enable_auto_scaling          = true
          max_count                    = 5
          max_pods                     = 30
          min_count                    = 1
          name                         = "default"
          only_critical_addons_enabled = false
          #orchestrator_version         = "1.20.7"
          vm_size                                = "Standard_D2_v2"
          os_disk_size_gb                    = 30
}
}

Answer 1

如前所述，您需要更改代码中的一些内容 in-order 才能使用 AKS 的流量管理器。

您需要使用 azureEndpoints 而不是 nestedEndpoints 作为流量管理器终结点类型。

因为目前有四种服务(Cloud Service ,App Service, App Service Slots and Public IP's)支持流量经理。所以，你必须使用 Public IP，它被 AKS .

您必须使用以下块：

resource "azurerm_traffic_manager_endpoint" "tmep" {
  name = "ansumanaks-tmep"
  resource_group_name = data.azurerm_resource_group.rg.name
  profile_name        = azurerm_traffic_manager_profile.tmp.name
  type                = "azureEndpoints"
  endpoint_status = "enabled"
  target_resource_id = (tolist(azurerm_kubernetes_cluster.k8s1.network_profile.0.load_balancer_profile.0.effective_outbound_ips)[0])
}

为了测试，我使用了下面的地形代码：

provider "azurerm" {
  features {}
}

data "azurerm_resource_group" "rg"{
    name="ansumantest"
}

# Traffic Manager Profile Resource
resource "azurerm_traffic_manager_profile" "tmp" {
  name = "ansumanaks-tm"
  resource_group_name = data.azurerm_resource_group.rg.name
  
  traffic_routing_method = "Priority"

  dns_config {
    relative_name = "ansumanaks-tm-dns-test"
    ttl           = 100
  }

  monitor_config {
    protocol                     = "http"
    port                         = 80
    path                         = "/"
    interval_in_seconds          = 30
    timeout_in_seconds           = 9
    tolerated_number_of_failures = 3
  }
}
resource "azurerm_public_ip" "example" {
  name                = "akspublicIP"
  resource_group_name = data.azurerm_resource_group.rg.name
  location            = data.azurerm_resource_group.rg.location
  sku = "Standard"
  allocation_method   = "Static"
  domain_name_label = "akstestregion"
}

# Traffic Manager Endpoint Resource
resource "azurerm_traffic_manager_endpoint" "tmep" {
  name = "ansumanaks-tmep"
  resource_group_name = data.azurerm_resource_group.rg.name
  profile_name        = azurerm_traffic_manager_profile.tmp.name
  type                = "azureEndpoints"
  endpoint_status = "enabled"
  target_resource_id = (tolist(azurerm_kubernetes_cluster.k8s1.network_profile.0.load_balancer_profile.0.effective_outbound_ips)[0])
}




################ K8S nodes pool location 1 ################

resource "azurerm_kubernetes_cluster" "k8s1" {
  name = "ansumanaks-k8s"
  location            = data.azurerm_resource_group.rg.location
  resource_group_name = data.azurerm_resource_group.rg.name
  dns_prefix          = "exampleaks1"

  service_principal {
    client_id = "1dd6833b-xxxx-xxxx-xxxx-112c3fb4fb79"
    client_secret = "e997Q~ky5ZWHIxxxxxxxxxxxxxxxxxxxxxxx"
  }
    role_based_access_control {
      azure_active_directory {
        managed           = true
        tenant_id         = "72f988bf-xxxx-xxxx-xxxx-2d7cd011db47"
        azure_rbac_enabled = true
      }
        enabled = true
    
    }   
    network_profile {
        network_plugin = "kubenet"
      load_balancer_profile {
        outbound_ip_address_ids= [azurerm_public_ip.example.id]
      }
    }

  linux_profile {
    admin_username = "ansuman"

    ssh_key {
      key_data = "${file("C:/Users/ansbal/public.pub")}"
    }
  }
auto_scaler_profile {
           new_pod_scale_up_delay           = "5s" 
           scale_down_delay_after_delete    = "10s" 
           skip_nodes_with_local_storage    = false
        }
addon_profile {
            azure_policy{
        enabled                    = true
      }
}


 default_node_pool {
          enable_auto_scaling          = true
          max_count                    = 5
          max_pods                     = 30
          min_count                    = 1
          name                         = "default"
          only_critical_addons_enabled = false
          #orchestrator_version        = "1.20.7"
          vm_size                      = "Standard_D2_v2"
          os_disk_size_gb              = 30
}
}

输出：

注：

我还创建了一个 Public IP，它将用于 AKS 负载平衡器，因为当我使用默认 public IP 进行测试时，它是在创建 AKS 时创建的，它给了我错误下面，为了解决我创建的 Public IP.

我已删除 admin_group_object_ids = [var.group_object_id] 由于缺少权限。您可以根据需要使用它们。

Terraform：使用流量管理器端点部署 Kubernetes 服务

Terraform: Deploy Kubernetes Services with Traffic Manager Endpoint

terraform

terraform-provider-azure

azure-rm