如何强制删除具有引用的 ADFPipeline

Question

我实际上是我的 ADF 的一些自动化。作为其中的一部分，我正在尝试删除所有 ADF V2 管道。问题是我的管道有很多对不同管道本身的引用。

$ADFPipeline = Get-AzDataFactoryV2Pipeline -DataFactoryName $(datafactory-name) -ResourceGroupName $(rg)



$ADFPipeline | ForEach-Object { Remove-AzDataFactoryV2Pipeline -ResourceGroupName $(rg) -DataFactoryName $(datafactory-name) -Name  $_.name -Force }

大多数时候我都会收到

这样的错误

The document cannot be deleted since it is referenced by "blabla"

我理解错误，它说一些引用并且不能删除。但是，当我尝试在 Azure 门户中进行相同的删除操作时，无论我可以删除的引用是什么。所以我想找到一种方法，即使它有引用也可以告诉 Powershell 是否可以强行删除它

非常感谢任何其他意见！

Answer 1

您好，谢谢您的提问。根据 the Remove-AzDataFactoryV2Pipeline doc，-Force 标志只是跳过确认提示。尽管有错误，它实际上并没有'Force'删除。

由于您已经在进行自动化，我建议您利用错误消息递归地尝试删除引用管道。 $error[0] 获取最近的错误。

（伪代码）

try_recurse_delete( pipeline_name )
    do_delete(pipeline_name)
    if not $error[0].contains("referenced by " + pipeline_name)
        then return true
    else
        try_recurse_delete( get_refrencer_name($error[0]) )

鉴于管道依赖关系可以是多对多关系，for-each 循环中的后续管道可能已被递归删除。您将不得不调整您的代码以对 'pipeline not found' 类型错误做出反应。

Answer 2

我运行遇到了同样的问题，发现从管道的 Activities 属性.

构建整个依赖关系图相当复杂

作为可行的解决方案 (powershell)：

function Remove-Pipelines {
    param (
        [Parameter(Mandatory=$true)]
        [AllowEmptyCollection()]
        [AllowNull()]
        [System.Collections.ArrayList]$pipelines
    )
    if($pipelines.Count -gt 0) {
        [System.Collections.ArrayList]$plsToProcess = New-Object System.Collections.ArrayList($null)
        foreach ($pipeline in $pipelines) { 
            try {
                $removeAzDFCommand = "Remove-AzDataFactoryV2Pipeline -dataFactoryName '$DataFactoryName' -resourceGroupName '$ResourceGroupName' -Name '$($pipeline.Name)' -Force -ErrorAction Stop"
                Write-Host $removeAzDFCommand
                Invoke-Expression $removeAzDFCommand
            }
            catch {
                if ($_ -match '.*The document cannot be deleted since it is referenced by.*') {
                    Write-Host $_
                    $plsToProcess.Add($pipeline)
                } else {
                    throw $_
                }
            }
        }
        Remove-Pipelines $plsToProcess
    }
}

这里是清除整个DF的完整解决方案："trigger","pipeline","dataflow","dataset","linkedService"

Param(
  [Parameter(Mandatory=$true)][string] $ResourceGroupName,
  [Parameter(Mandatory=$true)][string] $DataFactoryName
)

$artfTypes = "trigger","pipeline","dataflow","dataset","linkedService"

function Remove-Artifacts {
    param (
        [Parameter(Mandatory=$true)][AllowEmptyCollection()][AllowNull()][System.Collections.ArrayList]$artifacts,
        [Parameter(Mandatory=$true)][string]$artfType
    )
    if($artifacts.Count -gt 0) {
        [System.Collections.ArrayList]$artToProcess = New-Object System.Collections.ArrayList($null)
        foreach ($artifact in $artifacts) { 
            try {
                $removeAzDFCommand = "Remove-AzDataFactoryV2$($artfType) -dataFactoryName '$DataFactoryName' -resourceGroupName '$ResourceGroupName' -Name '$($artifact.Name)' -Force -ErrorAction Stop"
                Write-Host $removeAzDFCommand
                Invoke-Expression $removeAzDFCommand
            }
            catch {
                if ($_ -match '.*The document cannot be deleted since it is referenced by.*') {
                    Write-Host $_
                    $artToProcess.Add($artifact)
                } else {
                    throw $_
                }
            }
        }
        Remove-Artifacts $artToProcess $artfType
    }
}

foreach ($artfType in $artfTypes) {
  $getAzDFCommand = "Get-AzDataFactoryV2$($artfType) -dataFactoryName '$DataFactoryName' -resourceGroupName '$ResourceGroupName'"
  Write-Output $getAzDFCommand

  $artifacts = Invoke-Expression $getAzDFCommand
  Write-Output $artifacts.Name

  Remove-Artifacts $artifacts $artfType
}

同样的方法也适用于 "Set-AzDataFactoryV2Pipeline" 命令。

值得一提的是，随着依赖关系跟踪，Remove/Set 工件的顺序应该是正确的（因为跨工件的依赖关系）。

对于集合 - "linkedService","dataset","dataflow","pipeline","trigger"

删除 - "trigger","pipeline","dataflow","dataset","linkedService"

如何强制删除具有引用的 ADFPipeline

How to delete the ADFPipeline which is having the references Forcefully

powershell

pipeline

azure

azure-data-factory

azure-data-factory-2