Powershell、System.Diagnostics.Process 和 exiftool 在处理数百个命令时停止工作

Question

我创建了一个工具（准确地说：一个 Powershell 脚本）来帮助我转换文件夹中的图片，即它查找特定结尾的所有文件（例如，*.TIF）并通过以下方式将它们转换为 JPEG图片魔术。然后它通过 exiftool:

将一些 EXIF、IPTC 和 XMP 信息从源图像传输到 JPEG

# searching files (done before converting the files, so just listed for reproduction):
$WorkingFiles = @(Get-ChildItem -Path D:\MyPictures\Testfiles -Filter *.tif | ForEach-Object {
    [PSCustomObject]@{
        SourceFullName = $_.FullName
        JPEGFullName = $_.FullName -Replace 'tif$','jpg'
    }
})
# Then, converting is done. PowerShell will wait until every jpeg is successfully created.
# + + + + The problem occurs somewhere after this line + + + +
# Creating the exiftool process:
$psi = New-Object System.Diagnostics.ProcessStartInfo
$psi.FileName = .\exiftool.exe
$psi.Arguments = "-stay_open True -charset utf8 -@ -"
$psi.UseShellExecute = $false
$psi.RedirectStandardInput = $true
$psi.RedirectStandardOutput = $true
$psi.RedirectStandardError = $true
$exiftoolproc = [System.Diagnostics.Process]::Start($psi)

# creating the string argument for every file, then pass it over to exiftool:
for($i=0; $i -lt $WorkingFiles.length; $i++){
    [string]$ArgList = "-All:all=`n-charset`nfilename=utf8`n-tagsFromFile`n$($WorkingFiles[$i].SourceFullName)`n-EXIF:All`n-charset`nfilename=utf8`n$($WorkingFiles[$i].JPEGFullName)"
    # using -overwrite_original makes no difference
    # Also, just as good as above code:
    # [string]$ArgList = "-All:All=`n-EXIF:XResolution=300`n-EXIF:YResolution=300`n-charset`nfilename=utf8`n-overwrite_original`n$($WorkingFiles[$i].JPEGFullName)"

    $exiftoolproc.StandardInput.WriteLine("$ArgList`n-execute`n")
    # no difference using start-sleep:
    # Start-Sleep -Milliseconds 25
}
# close exiftool:
$exiftoolproc.StandardInput.WriteLine("-stay_open`nFalse`n")

# read StandardError and StandardOutput of exiftool, then print it:
[array]$outputerror = @($exiftoolproc.StandardError.ReadToEnd().Split("`r`n",[System.StringSplitOptions]::RemoveEmptyEntries))
[string]$outputout = $exiftoolproc.StandardOutput.ReadToEnd()
$outputout = $outputout -replace '========\ ','' -replace '\[1/1]','' -replace '\ \r\n\ \ \ \ '," - " -replace '{ready}\r\n',''
[array]$outputout = @($outputout.Split("`r`n",[System.StringSplitOptions]::RemoveEmptyEntries))

Write-Output "Errors:"
foreach($i in $outputerror){
    Write-Output $i
}
Write-Output "Standard output:"
foreach($i in $outputout){
    Write-Output $i
}

如果你想复现但又没有have/want那么多文件，还有一个更简单的方法：让exiftool打印出它的版本号600次：

$psi = New-Object System.Diagnostics.ProcessStartInfo
$psi.FileName = .\exiftool.exe
$psi.Arguments = "-stay_open True -charset utf8 -@ -"
$psi.UseShellExecute = $false
$psi.RedirectStandardInput = $true
$psi.RedirectStandardOutput = $true
$psi.RedirectStandardError = $true
$exiftoolproc = [System.Diagnostics.Process]::Start($psi)

for($i=0; $i -lt 600; $i++){
    try{
        $exiftoolproc.StandardInput.WriteLine("-ver`n-execute`n")
        Write-Output "Success:`t$i"
    }catch{
        Write-Output "Failed:`t$i"
    }
}
# close exiftool:
try{
    $exiftoolproc.StandardInput.WriteLine("-stay_open`nFalse`n")
}catch{
    Write-Output "Could not close exiftool!"
}

[array]$outputerror = @($exiftoolproc.StandardError.ReadToEnd().Split("`r`n",[System.StringSplitOptions]::RemoveEmptyEntries))
[array]$outputout = @($exiftoolproc.StandardOutput.ReadToEnd().Split("`r`n",[System.StringSplitOptions]::RemoveEmptyEntries))

Write-Output "Errors:"
foreach($i in $outputerror){
    Write-Output $i
}
Write-Output "Standard output:"
foreach($i in $outputout){
    Write-Output $i
}

据我测试，一切顺利，只要您保持< 115 个文件。如果你往上看，第 114^th JPEG 会获得正确的元数据，但 exiftool 在这之后停止工作 - 它空闲，我的脚本也是如此。我可以使用不同的文件、路径和 exiftool 命令重现它。

即使使用 exiftool 的 -verbose-flag，StandardOutput 和 StandardError 都没有显示任何异常 - 当然，他们不会，因为我必须杀死 exiftool 才能让他们出现。

运行 ISE 的/VSCode 的调试器没有显示任何内容。 Exiftool 的 window（仅在调试时出现）什么也没显示。

运行和 System.Diagnostics.Process 的命令是否有一些硬性限制，这是 exiftool 的问题还是仅仅是因为我无法使用最基本的 Powershell cmdlet 之外的东西？或者更好的问题是：我怎样才能正确调试它？

Powershell 是 5.1，exiftool 是 10.80（生产）- 10.94（最新）。

Answer 1

在弄乱了 $ArgList 的不同变体之后，我发现使用不同的文件命令没有区别，但是使用产生较少 StdOut 的命令（如 -ver）会导致更多迭代。因此，我有根据地猜测输出缓冲区是罪魁祸首。

As per Mark Byers' answer to "ProcessStartInfo hanging on “WaitForExit”? Why?":

The problem is that if you redirect StandardOutput and/or StandardError the internal buffer can become full. [...]

The solution is to use asynchronous reads to ensure that the buffer doesn't get full.

然后，这只是寻找合适的东西的问题。我发现 Alexander Obersht's answer to "How to capture process output asynchronously in powershell?" 几乎提供了我需要的一切。

脚本现在看起来像这样：

# searching files (done before converting the files, so just listed for reproduction):
$WorkingFiles = @(Get-ChildItem -Path D:\MyPictures\Testfiles -Filter *.tif | ForEach-Object {
    [PSCustomObject]@{
        SourceFullName = $_.FullName
        JPEGFullName = $_.FullName -Replace 'tif$','jpg'
    }
})
# Then, converting is done. PowerShell will wait until every jpeg is successfully created.
# Creating the exiftool process:
$psi = New-Object System.Diagnostics.ProcessStartInfo
$psi.FileName = .\exiftool.exe
$psi.Arguments = "-stay_open True -charset utf8 -@ -"
$psi.UseShellExecute = $false
$psi.RedirectStandardInput = $true
$psi.RedirectStandardOutput = $true
$psi.RedirectStandardError = $true

# + + + + NEW STUFF (1/2) HERE: + + + +
# Creating process object.
$exiftoolproc = New-Object -TypeName System.Diagnostics.Process
$exiftoolproc.StartInfo = $psi

# Creating string builders to store stdout and stderr.
$exiftoolStdOutBuilder = New-Object -TypeName System.Text.StringBuilder
$exiftoolStdErrBuilder = New-Object -TypeName System.Text.StringBuilder
# Adding event handers for stdout and stderr.
$exiftoolScripBlock = {
    if (-not [String]::IsNullOrEmpty($EventArgs.Data)){
        $Event.MessageData.AppendLine($EventArgs.Data)
    }
}
$exiftoolStdOutEvent = Register-ObjectEvent -InputObject $exiftoolproc -Action $exiftoolScripBlock -EventName 'OutputDataReceived' -MessageData $exiftoolStdOutBuilder
$exiftoolStdErrEvent = Register-ObjectEvent -InputObject $exiftoolproc -Action $exiftoolScripBlock -EventName 'ErrorDataReceived' -MessageData $exiftoolStdErrBuilder

[Void]$exiftoolproc.Start()
$exiftoolproc.BeginOutputReadLine()
$exiftoolproc.BeginErrorReadLine()
# + + + + END OF NEW STUFF (1/2) + + + +

# creating the string argument for every file, then pass it over to exiftool:
for($i=0; $i -lt $WorkingFiles.length; $i++){
    [string]$ArgList = "-All:all=`n-charset`nfilename=utf8`n-tagsFromFile`n$($WorkingFiles[$i].SourceFullName)`n-EXIF:All`n-charset`nfilename=utf8`n$($WorkingFiles[$i].JPEGFullName)"
    # using -overwrite_original makes no difference
    # Also, just as good as above code:
    # [string]$ArgList = "-All:All=`n-EXIF:XResolution=300`n-EXIF:YResolution=300`n-charset`nfilename=utf8`n-overwrite_original`n$($WorkingFiles[$i].JPEGFullName)"

    $exiftoolproc.StandardInput.WriteLine("$ArgList`n-execute`n")
}

# + + + + NEW STUFF (2/2) HERE: + + + +
# close exiftool:
$exiftoolproc.StandardInput.WriteLine("-stay_open`nFalse`n")
$exiftoolproc.WaitForExit()
# Unregistering events to retrieve process output.
Unregister-Event -SourceIdentifier $exiftoolStdOutEvent.Name
Unregister-Event -SourceIdentifier $exiftoolStdErrEvent.Name

# read StandardError and StandardOutput of exiftool, then print it:
[array]$outputerror = @($exiftoolStdErrBuilder.ToString().Trim().Split("`r`n",[System.StringSplitOptions]::RemoveEmptyEntries))
[string]$outputout = $exiftoolStdOutBuilder.ToString().Trim() -replace '========\ ','' -replace '\[1/1]','' -replace '\ \r\n\ \ \ \ '," - " -replace '{ready}\r\n',''
[array]$outputout = @($outputout.Split("`r`n",[System.StringSplitOptions]::RemoveEmptyEntries))
# + + + + END OF NEW STUFF (2/2) + + + +

Write-Output "Errors:"
foreach($i in $outputerror){
    Write-Output $i
}
Write-Output "Standard output:"
foreach($i in $outputout){
    Write-Output $i
}

我可以确认它适用于很多很多文件（至少 1600 个）。

Powershell、System.Diagnostics.Process 和 exiftool 在处理数百个命令时停止工作

Powershell, System.Diagnostics.Process & exiftool stop working when dealing with hundreds of commands

powershell

exiftool