运行 ForEach-Object -Parallel，导出时缺少数据

Question

我有一些工作代码基本上查询 2 个不同的 Graph API 端点，然后在 User Principal Name column 中搜索匹配项，并将 extension_335d4df9847945fbaa472c8b8fbb5d75_employeeNumber 列和值插入导出的csv（感谢用户@PMental 提供此解决方案）此列派生自最近从我们的本地 AD 扩展的属性。

这段代码工作得很好，但是如果我尝试并行化它，我在 extension_335d4df9847945fbaa472c8b8fbb5d75_employeeNumber 列中得不到任何结果。

这是因为一旦它被并行化，我就不能在并行进程之间共享变量了吗？如果是这样，我究竟该如何实现？

下面的代码 - 如果你删除 -Parallel，它工作正常：

$graphApiUri = "https://graph.microsoft.com/v1.0/reports/getOffice365ActiveUserDetail(period='D90')"

$Uri = "https://graph.microsoft.com/v1.0/users?`$select=userPrincipalName,extension_335d4df9847945fbaa472c8b8fbb5d75_employeeNumber"

$O365Report = Invoke-RestMethod -Method Get -Uri $graphApiUri -Headers $headerParams | ConvertFrom-Csv

# If the result is more than 999, we need to read the @odata.nextLink to show more than one side of users
$UserDetails = while (-not [string]::IsNullOrEmpty($uri)) {
    # API Call
    $apiCall = try {
        Invoke-RestMethod -Headers $headerParams -Uri $uri -Method Get
    }
    catch {
        $errorMessage = $_.ErrorDetails.Message | ConvertFrom-Json
    }
    $uri = $null
    if ($apiCall) {
        # Check if any data is left
        $uri = $apiCall.'@odata.nextLink'
        $apiCall
    }
}

Write-Output "Matching UPN to employeeNumber..."

$O365Report | ForEach-Object -Parallel {
    $CurrentEmpNumber = $UserDetails.value |
        Where-Object userPrincipalName -eq $_.'User Principal Name' |
            Select-Object -ExpandProperty extension_335d4df9847945fbaa472c8b8fbb5d75_employeeNumber -ErrorAction SilentlyContinue
    $_ | Add-Member -MemberType NoteProperty -Name extension_335d4df9847945fbaa472c8b8fbb5d75_employeeNumber -Value $CurrentEmpNumber
}

$O365Report | Export-Csv $ReportCSV -NoTypeInformation
Write-Output "Report saved to $ReportCSV."

Answer 1

当在 ForEach-Object -Parallel 脚本块内部时，您试图引用在其外部创建的变量，您需要在变量名称前加上 using:，这样它将是 $using:UserDetails

示例：

Returns 没什么，因为 $test 在并行脚本块的范围内不可访问：

$test = 1;
0..5 | % -Parallel { $test; };

Returns $test 的值五次，因为通过使用 $using:test 您现在可以看到它的值：

$test = 1;
0..5 | % -Parallel { $using:test; };

来自文档：

https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.1

The ForEach-Object -Parallel parameter set runs script blocks in parallel on separate process threads. The $using: keyword allows passing variable references from the cmdlet invocation thread to each running script block thread. Since the script blocks run in different threads, the object variables passed by reference must be used safely. Generally it is safe to read from referenced objects that don't change. But if the object state is being modified then you must used thread safe objects, such as .Net System.Collection.Concurrent types (See Example 11).

个人笔记：

我还建议使用 -ThrottleLimit 来限制其最大并行度。默认值为 5，但您可能需要更多或更少，具体取决于测试。

运行 ForEach-Object -Parallel，导出时缺少数据

Running ForEach-Object -Parallel, data missing from export

parallel-processing

powershell

office365

microsoft-graph-api

个人笔记：