使用 Azure Databricks 和 Terraform 时出错

Question

我的 Databricks 有以下代码。目前我只有工作区，但我的工作区中没有集群

  required_providers {
    azuread     = "~> 1.0"
    azurerm     = "~> 2.0"
    azuredevops = { source = "registry.terraform.io/microsoft/azuredevops", version = "~> 0.0" }
    databricks  = { source = "registry.terraform.io/databrickslabs/databricks", version = "~> 0.0" }
  }
}

provider "random" {}
provider "azuread" {
  tenant_id     = var.project.arm.tenant.id
  client_id     = var.project.arm.client.id
  client_secret = var.secret.arm.client.secret
}

provider "databricks" {
  host                        = azurerm_databricks_workspace.db-workspace.workspace_url
  azure_workspace_resource_id = azurerm_databricks_workspace.db-workspace.id
  azure_tenant_id             = var.project.arm.tenant.id
  azure_client_id             = var.project.arm.client.id
  azure_client_secret         = var.secret.arm.client.secret
}


resource "azurerm_databricks_workspace" "db-workspace" {
  name                          = module.names-db-workspace.environment.databricks_workspace.name_unique
  resource_group_name           = module.resourcegroup.resource_group.name
  location                      = module.resourcegroup.resource_group.location
  sku                           = "premium"
  public_network_access_enabled = true

  custom_parameters {
    no_public_ip                                         = true
    virtual_network_id                                   = module.virtualnetwork["centralus"].virtual_network.self.id
    public_subnet_name                                   = module.virtualnetwork["centralus"].virtual_network.subnets["db-sub-1-public"].name
    private_subnet_name                                  = module.virtualnetwork["centralus"].virtual_network.subnets["db-sub-2-private"].name
    public_subnet_network_security_group_association_id  = module.virtualnetwork["centralus"].virtual_network.nsgs.associations.subnets["databricks-public-nsg-db-sub-1-public"].id
    private_subnet_network_security_group_association_id = module.virtualnetwork["centralus"].virtual_network.nsgs.associations.subnets["databricks-private-nsg-db-sub-2-private"].id
  }
  tags = local.tags
}

Databricks 集群创建

resource "databricks_cluster" "dbcselfservice" {
  cluster_name            = format("adb-cluster-%s-%s", var.project.name, var.project.environment.name)
  spark_version           = var.spark_version
  node_type_id            = var.node_type_id
  autotermination_minutes = 20
  autoscale {
    min_workers = 1
    max_workers = 7
  }
  azure_attributes {
    availability       = "SPOT_AZURE"
    first_on_demand    = 1
    spot_bid_max_price = 100
  }
  depends_on = [
    azurerm_databricks_workspace.db-workspace
  ]
}

Databricks Workspace RBAC 权限

resource "databricks_group" "db-group" {
  display_name               = format("adb-users-%s", var.project.name)
  allow_cluster_create       = true
  allow_instance_pool_create = true
  depends_on = [
    resource.azurerm_databricks_workspace.db-workspace
  ]
}

resource "databricks_user" "dbuser" {
  count            = length(local.display_name)
  display_name     = local.display_name[count.index]
  user_name        = local.user_name[count.index]
  workspace_access = true
  depends_on = [
    resource.azurerm_databricks_workspace.db-workspace
  ]
}

将成员添加到 Databricks 管理员组

resource "databricks_group_member" "i-am-admin" {
  for_each  = toset(local.email_address)
  group_id  = data.databricks_group.admins.id
  member_id = databricks_user.dbuser[index(local.email_address, each.key)].id
  depends_on = [
    resource.azurerm_databricks_workspace.db-workspace
  ]
}

data "databricks_group" "admins" {
  display_name = "admins"
  depends_on = [
    #    resource.databricks_cluster.dbcselfservice,
    resource.azurerm_databricks_workspace.db-workspace
  ]
}

当我尝试运行 Terraform 计划时，出现以下错误：

Error: cannot read group: cannot configure azure-client-secret auth: cannot get workspace: please set `azure_workspace_resource_id` provider argument. Attributes used: azure_client_id, azure_client_secret, azure_tenant_id. Please check https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs#authentication for details
│ 
│   with databricks_group.db-group,
│   on resources.adb.tf line 71, in resource "databricks_group" "db-group":
│   71: resource "databricks_group" "db-group" {
│ 
╵
╷
│ Error: cannot read user: cannot configure azure-client-secret auth: cannot get workspace: please set `azure_workspace_resource_id` provider argument. Attributes used: azure_client_id, azure_client_secret, azure_tenant_id. Please check https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs#authentication for details
│ 
│   with databricks_user.dbuser[0],
│   on resources.adb.tf line 80, in resource "databricks_user" "dbuser":
│   80: resource "databricks_user" "dbuser" {

但是如果我在资源块“azurerm_databricks_workspace”中注释掉“custom_parameters”，我看不到错误。在 Azure 中，我只有 Databricks Workspace 而没有集群，我想创建集群并计划第二次运行 Terraform

几周前我删除并重新创建了我的子网。所以现在我的子网有了新名称

所以现在，如果我注释掉 custom_parameters，Terraform Apply 在集群创建时会抛出错误，并说它找不到旧的子网。但是我的子网引用在 custom_parameters 中，我不得不将其注释掉。

所以我现在处于 22 条军规。知道如何解决这个问题

Answer 1

您的代码几乎没有更改，请按照下面的建议进行更改。

在 db-group、dbuser、i-am-admin 和 admins 中使用 azurerm_databricks_workspace.db-workspace 而不是 resource.azurerm_databricks_workspace.db-workspace .

如本 Github 中所建议，尝试使用 Azurerm provider version 2.78 进行讨论，作为目前的解决方法，请先应用工作区创建，然后再应用其中的资源。

使用 Azure Databricks 和 Terraform 时出错

Getting Error with Azure Databricks and Terraform

azure

terraform-provider-azure

azure-databricks

terraform-provider-databricks