parquet 可以支持并发写操作吗?
Can parquet support concurrent write operations?
是否可以对 parquet 格式执行分布式并发写入?
是否可以在写入 parquet 文件时读取它们?
如果有并发的方法read/writes我有兴趣了解一下。
在此先感谢您的帮助。
我最终得到了 Parquet 开发人员的回答:两个问题的答案都是否:
Parquet writers are not thread-safe and files cannot be read or
written by different readers or writers concurrently. Parquet doesn't
expose flush/sync operations to the user (for good reason) so there
isn't a way to reliably do this anyway.
是否可以对 parquet 格式执行分布式并发写入?
是否可以在写入 parquet 文件时读取它们?
如果有并发的方法read/writes我有兴趣了解一下。
在此先感谢您的帮助。
我最终得到了 Parquet 开发人员的回答:两个问题的答案都是否:
Parquet writers are not thread-safe and files cannot be read or written by different readers or writers concurrently. Parquet doesn't expose flush/sync operations to the user (for good reason) so there isn't a way to reliably do this anyway.