GNU Parallel 作为作业队列——未执行的最后命令

Question

尝试使用 GNU parallel 20201222 跟随 GNU Parallel as job queue with named pipes，我运行遇到了 parallel 的问题，不执行通过 tail -n+0 -f 管道输入的最后命令。

为了演示，我打开了 3 个终端：

# terminal 1
true > jobqueue
tail -n+0 -f jobqueue | parallel

# terminal 2
tail -n+0 -f jobqueue | cat

将单个小测试命令添加到队列中：

# terminal 3
echo "echo test" >> jobqueue

只有终端2打印“echo test”，gnu parallel不输出任何东西。

# terminal 3
for i in `seq 10`; do echo "echo $i" >> jobqueue; done

仅终端 2 打印“echo 1”，...，“echo 10”（每行一个），gnu parallel 不输出任何内容。

# terminal 3
for i in `seq 100`; do echo "echo $i" >> jobqueue; done

2 号终端打印“echo 1”，...，“echo 100”。终端 1 打印“test”、“1”、...、“10”、“1”、...、“99”行，最后一行“100”丢失。

Re运行ning tail -n+0 -f jobqueue | parallel 输出全部到“99”。重新运行附加 --resume --joblog log 后，再输出一行（“100”），但一旦新行添加到 joblog 也会落后。对于 GNU parallel 20161222，初始运行仅到达“84”行。

如何强制 gnu parallel 在每一行刷新其输入队列？

Answer 1

来自man parallel：

There is a a small issue when using GNU parallel as queue system/batch manager: You have to submit JobSlot number of jobs before they will start, and after that you can submit one at a time, and job will start immediately if free slots are available. Output from the running or completed jobs are held back and will only be printed when JobSlots more jobs has been started (unless you use --ungroup or --line-buffer, in which case the output from the jobs are printed immediately). E.g. if you have 10 jobslots then the output from the first completed job will only be printed when job 11 has started, and the output of second completed job will only be printed when job 12 has started.

换句话说：职位是运行。输出延迟。如果您在示例中使用 touch unique-file-name.

而不是使用 echo 则更容易看出

GNU Parallel 作为作业队列——未执行的最后命令

GNU Parallel as job queue -- last commands not executed

gnu-parallel