UnsupportedOperationException 在 IoTDB 中的 Compaction Merge 中发生,当 TsFile 存储在 HDFS 中时
UnsupportedOperationException occurred in Compaction Merge in IoTDB when TsFile storage in HDFS
我尝试将 TSFiles 本地化到 HDFS,配置如下:
tsfile_storage_fs=HDFS
core_site_path=/home/hadoop-2.7.7/etc/hadoop/core-site.xml
hdfs_site_path=/home/hadoop-2.7.7/etc/hadoop/hdfs-site.xml
hdfs_ip=IP
hdfs_port=port
异常是:
2021-08-10 14:23:28,262 [pool-4-IoTDB-Compaction-10] ERROR o.a.i.d.e.c.l.LevelCompactionTsFileManagement:740 - Error occurred in Compaction Merge thread java.lang.UnsupportedOperationException: Unsupported operation.
at org.apache.iotdb.hadoop.fileSystem.HDFSFile.getParent(HDFSFile.java:293) at org.apache.iotdb.db.engine.storagegroup.TsFileResource.modifyTsFileNameMergeCnt(TsFileResource.java:822)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:668)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:596)
at org.apache.iotdb.db.engine.compaction.TsFileManagement$CompactionMergeTask.call(TsFileManagement.java:172)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.syncCompactOnePartition(StorageGroupProcessor.java:2003)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.access0(StorageGroupProcessor.java:137)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor$CompactionAllPartitionTask.call(StorageGroupProcessor.java:527)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor$CompactionAllPartitionTask.call(StorageGroupProcessor.java:514)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access1(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2021-08-10 14:23:28,262 [pool-4-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:744 - root.vehicle [Compaction] merge end time isSeq = true, consumption: 928 ms
IoTDB 的压缩还没有适应 HDFS 中的存储。使用 HDFS 时,请通过设置 compaction_strategy=NO_COMPACTION
和 enable_unseq_compaction=false
关闭压缩。
我尝试将 TSFiles 本地化到 HDFS,配置如下:
tsfile_storage_fs=HDFS
core_site_path=/home/hadoop-2.7.7/etc/hadoop/core-site.xml
hdfs_site_path=/home/hadoop-2.7.7/etc/hadoop/hdfs-site.xml
hdfs_ip=IP
hdfs_port=port
异常是:
2021-08-10 14:23:28,262 [pool-4-IoTDB-Compaction-10] ERROR o.a.i.d.e.c.l.LevelCompactionTsFileManagement:740 - Error occurred in Compaction Merge thread java.lang.UnsupportedOperationException: Unsupported operation.
at org.apache.iotdb.hadoop.fileSystem.HDFSFile.getParent(HDFSFile.java:293) at org.apache.iotdb.db.engine.storagegroup.TsFileResource.modifyTsFileNameMergeCnt(TsFileResource.java:822)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:668)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:596)
at org.apache.iotdb.db.engine.compaction.TsFileManagement$CompactionMergeTask.call(TsFileManagement.java:172)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.syncCompactOnePartition(StorageGroupProcessor.java:2003)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.access0(StorageGroupProcessor.java:137)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor$CompactionAllPartitionTask.call(StorageGroupProcessor.java:527)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor$CompactionAllPartitionTask.call(StorageGroupProcessor.java:514)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access1(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2021-08-10 14:23:28,262 [pool-4-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:744 - root.vehicle [Compaction] merge end time isSeq = true, consumption: 928 ms
IoTDB 的压缩还没有适应 HDFS 中的存储。使用 HDFS 时,请通过设置 compaction_strategy=NO_COMPACTION
和 enable_unseq_compaction=false
关闭压缩。