Apache Kylin - 立方体构建第一步的错误输出

Question

我正在尝试使用 Apache Kylin 构建我的第一个多维数据集，一切正常，直到我遇到错误的最后一步：

java.lang.IllegalStateException: Can't get cube source record count.
at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
at org.apache.kylin.job.cube.UpdateCubeInfoAfterBuildStep.doWork(UpdateCubeInfoAfterBuildStep.java:104)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

根据这个问题 https:// github.com/KylinOLAP/Kylin/issues/101 上面的错误发生是因为 Kylin 试图在 hive 的输出中找到这个模式："HDFS Read: (\d+) HDFS Write: (\d+) SUCCESS".

立方体构建教程的正确输出：github。com/KylinOLAP/Kylin/wiki/Kylin-Cube-Build-and-Job-Monitoring-Tutorial：

http://i.imgur.com/71tEeIy.png

我在 Kylin 中的输出看起来损坏了：

http://i.imgur.com/GCbrN03.png

然而，当我检查 Hive 日志时，它看起来没问题：

2015-05-27 08:40:13,419 INFO  [main]: ql.Driver (Driver.java:execute(1285)) - Starting command: 
INSERT OVERWRITE TABLE kylin_intermediate_Kubek_19700101000000_2922789940817071255_f23ac1b1_10fe_4112_ac9e_b4e6baf07654 SELECT
FACT_TABLE.DATE
,FACT_TABLE.MONEY_ADVERTISER
,FACT_TABLE.MONEY_PUBLISHER
FROM DEFAULT.ADVSTATS as FACT_TABLE 
...
2015-05-27 08:45:05,132 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - MapReduce Jobs Launched: 
2015-05-27 08:45:05,148 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - Stage-Stage-1: Map: 1   Cumulative CPU: 17.32 sec   HDFS Read: 44644035 HDFS Write: 2347008 SUCCESS
2015-05-27 08:45:05,153 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - Total MapReduce CPU Time Spent: 17 seconds 320 msec
2015-05-27 08:45:05,167 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - OK

我正在使用 Hortonworks Sandbox 2.2：

hadoop-2.6.0
hbase-0.98.12
hive-0.14.0
zookeeper-3.4.6

谁能告诉我为什么我的 kylin 日志预览看起来像那样，首先这会不会是最后一步错误的原因？

Answer 1

前几天kylin开发者针对此类问题提交了绕过

https://github.com/apache/incubator-kylin/commit/a4692dba681bc2f136e02c64565639eb0080fcc9

因为有时候hadoop可能会获取counter失败，即使job成功了从现在开始，Kylin 在获取 cube 源失败时给出警告而不是错误。

我所要做的就是重建麒麟

Apache Kylin - 立方体构建第一步的错误输出

Apache Kylin - wrong output at the first step of cube building

apache

hadoop

hive

olap

kylin