Kettle (Pentaho PDI):并发写入文本文件输出

Kettle (Pentaho PDI): concurrent write with Text File Output

显然,Kettle的Text File Output步骤不支持并发:如果两个不同的Text File Output步骤写到同一个文件,没有实现锁定机制,因此生成的输出文件包含(错误的)输出行,这些行是由输入源的字段混合产生的。任何人都可以向我提供此行为的确认信息(如果有可信来源则更好)?

摘自本书第 387 页Pentaho Kettle Solutions

Writing to a single text file in parallel is not possible. For example, you can’t use multiple step copies of the “Text file output” step to write to the same output file. If you try it, the result is a blended file with lines and fields ending up in the wrong location. This problem can only be countered with advanced locking algorithms that in turn again reduce the degree of parallelism to a single thread. However, the simple solution is to write to multiple output files.

当您通过多个 "Text file Output" 步骤写入同一个文件时,就好像该文件被两个具有不同上下文的独立程序访问一样。所以,在这种情况下,Pentaho 当然不会处理并发。 "Text file Output" 中的文件写入函数虽然是同步的,但它在单个上下文中处理并发。