使用 Terraform Kubernetes 提供程序向新创建的 Azure AKS 添加机密失败

Question

我正在使用 Azure Terraform 提供程序创建一个 kubernetes 集群并尝试向其添加一个秘密。集群创建得很好，但在创建密钥时我在对集群进行身份验证时遇到错误。我尝试了 2 种不同的 Terraform Kubernetes 提供程序配置。这是主要配置：

variable "client_id" {}
variable "client_secret" {}

resource "azurerm_resource_group" "rg-example" {
  name     = "rg-example"
  location = "East US"
}

resource "azurerm_kubernetes_cluster" "k8s-example" {
  name                = "k8s-example"
  location            = azurerm_resource_group.rg-example.location
  resource_group_name = azurerm_resource_group.rg-example.name
  dns_prefix          = "k8s-example"

  default_node_pool {
    name            = "default"
    node_count      = 1
    vm_size         = "Standard_B2s"
  }

  service_principal {
    client_id     = var.client_id
    client_secret = var.client_secret
  }

  role_based_access_control {
    enabled = true
  }
}

resource "kubernetes_secret" "secret_example" {
  metadata {
    name = "mysecret"
  }
  data = {
    "something" = "super secret"
  }
  depends_on = [
    azurerm_kubernetes_cluster.k8s-example
  ]
}

provider "azurerm" {
  version = "=2.29.0"
  features {}
}

output "host" {
  value = azurerm_kubernetes_cluster.k8s-example.kube_config.0.host
}
output "cluster_username" {
  value = azurerm_kubernetes_cluster.k8s-example.kube_config.0.username
}
output "cluster_password" {
  value = azurerm_kubernetes_cluster.k8s-example.kube_config.0.password
}
output "client_key" {
  value = azurerm_kubernetes_cluster.k8s-example.kube_config.0.client_key
}
output "client_certificate" {
  value = azurerm_kubernetes_cluster.k8s-example.kube_config.0.client_certificate
}
output "cluster_ca_certificate" {
  value = azurerm_kubernetes_cluster.k8s-example.kube_config.0.cluster_ca_certificate
}

这是第一个使用证书的 Kubernetes 供应商配置：

provider "kubernetes" {
  version = "=1.13.2"
  load_config_file = "false"

  host = azurerm_kubernetes_cluster.k8s-example.kube_config.0.host
  
  client_certificate     = azurerm_kubernetes_cluster.k8s-example.kube_config.0.client_certificate
  client_key             = azurerm_kubernetes_cluster.k8s-example.kube_config.0.client_key
  cluster_ca_certificate = azurerm_kubernetes_cluster.k8s-example.kube_config.0.cluster_ca_certificate
}

我收到的错误是：

kubernetes_secret.secret_example: Creating...

Error: Failed to configure client: tls: failed to find any PEM data in certificate input

这是使用 HTTP 基本授权的第二个 Kubernetes 提供程序配置：

provider "kubernetes" {
  version = "=1.13.2"
  load_config_file = "false"

  host = azurerm_kubernetes_cluster.k8s-example.kube_config.0.host
  
  username = azurerm_kubernetes_cluster.k8s-example.kube_config.0.username
  password = azurerm_kubernetes_cluster.k8s-example.kube_config.0.password
}

我收到的错误是：

kubernetes_secret.secret_example: Creating...

Error: Post "https://k8s-example-c4a78c03.hcp.eastus.azmk8s.io:443/api/v1/namespaces/default/secrets": x509: certificate signed by unknown authority

分析

我检查了 azurerm_kubernetes_cluster.k8s-example 的输出，数据似乎有效（用户名、密码、主机等）。也许我的 Kubernetes 集群需要一个 SSL 证书，但是我不确定，因为我是新手。有人可以帮我吗？

Answer 1

根据this issue in hashicorp/terraform-provider-kubernetes，您需要使用base64decode()。作者使用的例子：

provider "kubernetes" {
  host = "${google_container_cluster.k8sexample.endpoint}"
  username = "${var.master_username}"
  password = "${var.master_password}"
  client_certificate = "${base64decode(google_container_cluster.k8sexample.master_auth.0.client_certificate)}"
  client_key = "${base64decode(google_container_cluster.k8sexample.master_auth.0.client_key)}"
  cluster_ca_certificate = "${base64decode(google_container_cluster.k8sexample.master_auth.0.cluster_ca_certificate)}"
}

那个作者说，如果他们遗漏了 base64 解码，他们就会得到和你一样的错误。您可以在此处阅读有关该功能的更多信息：https://www.terraform.io/docs/configuration/functions/base64decode.html

使用 Terraform Kubernetes 提供程序向新创建的 Azure AKS 添加机密失败

Add secret to freshly created Azure AKS using Terraform Kubernetes provider fails

certificate

kubernetes

terraform

terraform-provider-azure

azure-aks