cassandra 上的 Nutch 2.3.1 无法启动

Question

我正在尝试运行将 2.3.1 与 cassandra 结合起来。按照 http://wiki.apache.org/nutch/Nutch2Cassandra 上的步骤操作。最后，当我尝试使用命令启动 nutch 时：

bin/crawl urls/ test http://localhost:8983/solr/ 2

我遇到以下异常：

GeneratorJob: starting
GeneratorJob: filtering: false
GeneratorJob: normalizing: false
GeneratorJob: topN: 50000
GeneratorJob: java.lang.RuntimeException: job failed: name=[test]generate: 1454483370-31180, jobid=job_local1380148534_0001
    at     org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:120)
    at org.apache.nutch.crawl.GeneratorJob.run(GeneratorJob.java:227)
    at org.apache.nutch.crawl.GeneratorJob.generate(GeneratorJob.java:256)
    at org.apache.nutch.crawl.GeneratorJob.run(GeneratorJob.java:322)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.nutch.crawl.GeneratorJob.main(GeneratorJob.java:330)

Error running:
  /home/user/apache-nutch-2.3.1/runtime/local/bin/nutch generate -D    mapred.reduce.tasks=2 -D mapred.child.java.opts=-Xmx1000m -D mapred.reduce.tasks.speculative.execution=false -D mapred.map.tasks.speculative.execution=false -D mapred.compress.map.output=true -topN 50000 -noNorm -noFilter -adddays 0 -    crawlId webmd -batchId 1454483370-31180
Failed with exit value 255.

当我检查 logs/hadoop.log 时，错误消息如下：

2016-02-03 15:18:14,741 ERROR connection.HConnectionManager - Could not start connection pool for host localhost(127.0.0.1):9160
...
2016-02-03 15:18:15,185 ERROR store.CassandraStore - All host pools marked down. Retry burden pushed out to client.
me.prettyprint.hector.api.exceptions.HectorException: All host pools marked down. Retry burden pushed out to client.
    at me.prettyprint.cassandra.connection.HConnectionManager.getClientFromLBPolicy(HConnectionManager.java:390)

但是我的 cassandra 服务器已启动：

runtime/local$ netstat -l |grep 9160
tcp        0      0 172.16.230.130:9160     *:*                     LISTEN

有人可以帮助解决这个问题吗？谢谢。

Answer 1

Cassandra 的地址不是 localhost，而是 172.16.230.130。这就是 Nutch 无法连接到 Cassandra 商店的原因。

希望这对您有所帮助，

乐国岛

cassandra 上的 Nutch 2.3.1 无法启动

Nutch 2.3.1 on cassandra couldn't start

hadoop

nutch

cassandra