Apache phoenix 并发查询因异常而失败

Question

我们正在尝试一系列操作，例如 SELECT -> 将行键存储到集合中 -> 然后将集合拆分为每个工作线程 -> 每个线程再次使用 phoenix jdbc 创建连接-> 执行 SELECT 然后根据结果 UPSERT 到不同的凤凰 table.

我正在使用固定线程池 4 的 ExecutorService 我看到如下异常。

org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: The system cannot find the path specified
       at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
       at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:538)
       at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
       at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
       at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
       at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:764)
       at com.vonage.test.PopulateStagingGWCDRWorker.run(MyCode.java:74)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: The system cannot find the path specified
       at java.util.concurrent.FutureTask.report(FutureTask.java:122)
       at java.util.concurrent.FutureTask.get(FutureTask.java:206)
       at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:534)
       ... 8 more
Caused by: org.apache.phoenix.exception.PhoenixIOException: The system cannot find the path specified
       at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
       at org.apache.phoenix.iterate.SpoolingResultIterator.<init>(SpoolingResultIterator.java:122)
       at org.apache.phoenix.iterate.SpoolingResultIterator.<init>(SpoolingResultIterator.java:73)
       at org.apache.phoenix.iterate.SpoolingResultIterator$SpoolingResultIteratorFactory.newIterator(SpoolingResultIterator.java:67)
       at org.apache.phoenix.iterate.ChunkedResultIterator.<init>(ChunkedResultIterator.java:92)
       at org.apache.phoenix.iterate.ChunkedResultIterator$ChunkedResultIteratorFactory.newIterator(ChunkedResultIterator.java:72)
       at org.apache.phoenix.iterate.ParallelIterators.call(ParallelIterators.java:92)
       at org.apache.phoenix.iterate.ParallelIterators.call(ParallelIterators.java:83)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       ... 3 more
Caused by: java.io.IOException: The system cannot find the path specified
       at java.io.WinNTFileSystem.createFileExclusively(Native Method)
       at java.io.File.createTempFile(File.java:2024)
       at org.apache.commons.io.output.DeferredFileOutputStream.thresholdReached(DeferredFileOutputStream.java:176)
       at org.apache.phoenix.iterate.SpoolingResultIterator.thresholdReached(SpoolingResultIterator.java:98)
       at org.apache.commons.io.output.ThresholdingOutputStream.checkThreshold(ThresholdingOutputStream.java:224)
       at org.apache.commons.io.output.ThresholdingOutputStream.write(ThresholdingOutputStream.java:92)
       at java.io.DataOutputStream.writeByte(DataOutputStream.java:153)
       at org.apache.hadoop.io.WritableUtils.writeVLong(WritableUtils.java:273)
       at org.apache.hadoop.io.WritableUtils.writeVInt(WritableUtils.java:253)
       at org.apache.phoenix.util.TupleUtil.write(TupleUtil.java:146)
       at org.apache.phoenix.iterate.SpoolingResultIterator.<init>(SpoolingResultIterator.java:107)
       ... 10 more
enter code here

但是如果我使用的池大小为 2 或更小，它就可以正常工作。我想知道客户端是否有可以更改的属性？

Answer 1

hbase-site.xml 中的

phoenix.spool.directory 修复了此问题。谢谢

Answer 2

我的情况是通过在 pom.xml

中使用以下依赖项解决了这个问题

<dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-protocol</artifactId>
            <version>1.1.11</version>
</dependency>

更新一下，我的 Hbase 版本是：1.1，phoenix 是 4.7

Apache phoenix 并发查询因异常而失败

Apache phoenix concurrent queries failing with exception

java

phoenix