同步使用 Add-Content 的 PowerShell 后台作业

Question

我有一个小型 PowerShell 程序，它启动几个线程来进行并行计算，然后在它们完成后将包含结果的一行附加到文本文件中，然后继续执行更多操作。这在开发和测试中运行良好，但在生产中偶尔会挂起，而且文件似乎是 "jammed open"。我将写入包裹在 "try" 块中，但这无济于事。我写了一个玩具应用程序来说明这个问题，它通常在大约 10-15 分钟后挂起（写了大约 3000 行）。
在我看来，如果 Python 解决方案使用互斥锁或其他东西，我会过得更好，但我现在在这条路上走得很远。寻找想法如何轻松解决此问题。我真的以为 Add-Content 会是原子的...

Parentjob.ps1

# Start a bunch of jobs

$curdir = "c:\transfer\filecollide"
$tokens = "tok00","tok01","tok02",
          "tok03","tok04","tok05",
          "tok06","tok07","tok08"

$jobs = @()
foreach ($tok in $tokens)
{
  $job = Start-Job -FilePath ".\childjob.ps1" -ArgumentList "${curdir}",$tok,2,1000
  Start-Sleep -s 3  # stagger things a bit
  Write-Output "    Starting:${tok} job"
  $jobs += ,$job
}

foreach ($job in $jobs)
{
  wait-job $job 
  $out = receive-job $job
  Write-Output($out)
}

childjob.ps1

param(
    [string]$curdir = ".",
    [string]$tok = "tok?",
    [int]$interval = 10,
    [int]$ntodo = 1
)

$nwritefails = 0
$nwritesuccess = 0
$nwrite2fails = 0

function singleLine
{
    param(
        [string]$tok,
        [string]$fileappendout = "",
        [int]$timeout = 3
    )

    $curdatetime = (Get-Date)
    $sout = "${curdatetime},${tok},${global:nwritesuccess},${global:nwritefails},${global:nwrite2fails}"
    $global:nwritesuccess++
    try
    {
         Add-Content -Path $fileappendout -Value "${sout}"
    }
    catch
    {
         $global:nwritefails++
         try
         {
            Start-Sleep -s 1
            Add-Content -Path $fileappendout -Value "${sout}"
         }
         catch
         {
            $global:nwrite2fails++
            Write-Output "Failed to write to ${fileappendout}"
         }
    }
}


Write-Output "Starting to process ${tok}"
#Start of main code

cd "${curdir}"
$ndone = 0

while ($true)
{
    singleLine $tok "outfile.txt" 
    $ndone++
    if ($ndone -gt $ntodo){ break }
    Start-Sleep -s $interval
}
Write-Output "Successful ${tok} appends:${nwritesuccess}  failed:${nwritefails} failed2:${nwrite2fails}"

Answer 1

为什么不让作业将结果写入输出流，并在主线程中使用 Receive-Job 来收集结果并更新文件？您可以在作业仍然运行时执行此操作。您写入输出流的内容现在看起来可能更适合写入 Progress 流。

同步使用 Add-Content 的 PowerShell 后台作业

Synchronising PowerShell background jobs that use Add-Content

powershell

file-io

background-process