Hadoop map/reduce shows Error: com.mongodb.connection.SslSettings$Builder.context(Ljavax/net/ssl/SSLContext;)
Hadoop map/reduce shows Error: com.mongodb.connection.SslSettings$Builder.context(Ljavax/net/ssl/SSLContext;)
我 运行 map/reduce 定期工作，最近我收到了这样的日志 :
17/09/27 10:38:03 INFO mapreduce.Job: map 27% reduce 0%
17/09/27 10:38:07 INFO mapreduce.Job: Task Id :
attempt_1494368272823_5583_m_000014_0, Status : FAILED
Error: com.mongodb.connection.SslSettings$Builder.context(Ljavax/net/ssl/SSLContext;)Lcom/mongodb/connection/SslSettings$Builder;
17/09/27 10:38:08 INFO mapreduce.Job: map 30% reduce 0%
17/09/27 10:38:10 INFO mapreduce.Job: map 30% reduce 6%
17/09/27 10:38:43 INFO mapreduce.Job: map 32% reduce 6%
17/09/27 10:38:45 INFO mapreduce.Job: map 33% reduce 6%
17/09/27 10:38:46 INFO mapreduce.Job: map 33% reduce 7%
17/09/27 10:38:49 INFO mapreduce.Job: map 35% reduce 7%
17/09/27 10:38:52 INFO mapreduce.Job: map 35% reduce 8%
17/09/27 10:39:15 INFO mapreduce.Job: Task Id : attempt_1494368272823_5583_m_000008_0, Status : FAILED
Error: com.mongodb.connection.SslSettings$Builder.context(Ljavax/net/ssl/SSLContext;)Lcom/mongodb/connection/SslSettings$Builder;
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
17/09/27 10:39:16 INFO mapreduce.Job: map 40% reduce 10%
结尾为：
17/09/27 10:44:12 INFO mapreduce.Job: Job job_1494368272823_5583 failed with state FAILED due to: Task failed task_1494368272823_5583_m_000025
Job failed as tasks failed. failedMaps:1 failedReduces:0
17/09/27 10:44:12 INFO mapreduce.Job: Counters: 41
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=5481350
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=3430808431
HDFS: Number of bytes written=2495944
HDFS: Number of read operations=5540
HDFS: Number of large read operations=0
HDFS: Number of write operations=118
Job Counters
Failed map tasks=21
Killed map tasks=9
Killed reduce tasks=1
Launched map tasks=47
Launched reduce tasks=1
Other local map tasks=20
Data-local map tasks=26
Rack-local map tasks=1
Total time spent by all maps in occupied slots (ms)=34635900
Total time spent by all reduces in occupied slots (ms)=3707820
Total time spent by all map tasks (ms)=3463590
Total time spent by all reduce tasks (ms)=370782
Total vcore-milliseconds taken by all map tasks=3463590
Total vcore-milliseconds taken by all reduce tasks=370782
Total megabyte-milliseconds taken by all map tasks=34635900000
Total megabyte-milliseconds taken by all reduce tasks=3707820000
Map-Reduce Framework
Map input records=18
Map output records=9722
Map output bytes=2572358
Map output materialized bytes=2608684
Input split bytes=3387
Combine input records=0
Spilled Records=9722
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=62408
CPU time spent (ms)=1809020
Physical memory (bytes) snapshot=56575217664
Virtual memory (bytes) snapshot=155695132672
Total committed heap usage (bytes)=60439920640
File Input Format Counters
Bytes Read=0
MapReduce failed with 1
有时作业实际上会成功完成。有时就像上面那样。日志中没有提供更多信息。对于导致此失败的到底发生了什么，我将不胜感激？据我所知，这似乎不是超时或内存问题。集群没有超载（我检查过）...
编辑：进一步调试后，当我连接到 MongoDB 时，问题似乎出现在 "connect" 方法中。但是为什么会这样呢？当我尝试手动连接时连接成功，并且有时它可以工作。
我想你使用了一个 MongoClient 实例。
在这种情况下，请验证与 SSLContext 相关的参数是否按照以下参考中所述进行了设置：
http://mongodb.github.io/mongo-java-driver/3.6/driver/tutorials/ssl/
"Java Secure Socket Extension (JSSE) Reference Guide" 将是了解更多 SSLContext 的好书：
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html
您会发现大量关于创建 SSLContext.
实例的示例
否则，您很可能 运行 陷入了这个问题：
https://jira.mongodb.org/browse/JAVA-2590
摘自link：
an older version of mongodb-driver-core is being pulled in
请清理你的依赖并使用最新的。
我 运行 map/reduce 定期工作，最近我收到了这样的日志 :
17/09/27 10:38:03 INFO mapreduce.Job: map 27% reduce 0%
17/09/27 10:38:07 INFO mapreduce.Job: Task Id :
attempt_1494368272823_5583_m_000014_0, Status : FAILED Error: com.mongodb.connection.SslSettings$Builder.context(Ljavax/net/ssl/SSLContext;)Lcom/mongodb/connection/SslSettings$Builder;
17/09/27 10:38:08 INFO mapreduce.Job: map 30% reduce 0%
17/09/27 10:38:10 INFO mapreduce.Job: map 30% reduce 6%
17/09/27 10:38:43 INFO mapreduce.Job: map 32% reduce 6%
17/09/27 10:38:45 INFO mapreduce.Job: map 33% reduce 6%
17/09/27 10:38:46 INFO mapreduce.Job: map 33% reduce 7%
17/09/27 10:38:49 INFO mapreduce.Job: map 35% reduce 7%
17/09/27 10:38:52 INFO mapreduce.Job: map 35% reduce 8%17/09/27 10:39:15 INFO mapreduce.Job: Task Id : attempt_1494368272823_5583_m_000008_0, Status : FAILED Error: com.mongodb.connection.SslSettings$Builder.context(Ljavax/net/ssl/SSLContext;)Lcom/mongodb/connection/SslSettings$Builder;
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 14317/09/27 10:39:16 INFO mapreduce.Job: map 40% reduce 10%
结尾为：
17/09/27 10:44:12 INFO mapreduce.Job: Job job_1494368272823_5583 failed with state FAILED due to: Task failed task_1494368272823_5583_m_000025 Job failed as tasks failed. failedMaps:1 failedReduces:0
17/09/27 10:44:12 INFO mapreduce.Job: Counters: 41 File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=5481350
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=3430808431
HDFS: Number of bytes written=2495944
HDFS: Number of read operations=5540
HDFS: Number of large read operations=0
HDFS: Number of write operations=118 Job Counters
Failed map tasks=21
Killed map tasks=9
Killed reduce tasks=1
Launched map tasks=47
Launched reduce tasks=1
Other local map tasks=20
Data-local map tasks=26
Rack-local map tasks=1
Total time spent by all maps in occupied slots (ms)=34635900
Total time spent by all reduces in occupied slots (ms)=3707820
Total time spent by all map tasks (ms)=3463590
Total time spent by all reduce tasks (ms)=370782
Total vcore-milliseconds taken by all map tasks=3463590
Total vcore-milliseconds taken by all reduce tasks=370782
Total megabyte-milliseconds taken by all map tasks=34635900000
Total megabyte-milliseconds taken by all reduce tasks=3707820000
Map-Reduce Framework
Map input records=18
Map output records=9722
Map output bytes=2572358
Map output materialized bytes=2608684
Input split bytes=3387
Combine input records=0
Spilled Records=9722
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=62408
CPU time spent (ms)=1809020
Physical memory (bytes) snapshot=56575217664
Virtual memory (bytes) snapshot=155695132672
Total committed heap usage (bytes)=60439920640
File Input Format Counters
Bytes Read=0
MapReduce failed with 1
有时作业实际上会成功完成。有时就像上面那样。日志中没有提供更多信息。对于导致此失败的到底发生了什么，我将不胜感激？据我所知，这似乎不是超时或内存问题。集群没有超载（我检查过）...
编辑：进一步调试后，当我连接到 MongoDB 时，问题似乎出现在 "connect" 方法中。但是为什么会这样呢？当我尝试手动连接时连接成功，并且有时它可以工作。
我想你使用了一个 MongoClient 实例。
在这种情况下，请验证与 SSLContext 相关的参数是否按照以下参考中所述进行了设置：
http://mongodb.github.io/mongo-java-driver/3.6/driver/tutorials/ssl/
"Java Secure Socket Extension (JSSE) Reference Guide" 将是了解更多 SSLContext 的好书：
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html
您会发现大量关于创建 SSLContext.
否则，您很可能 运行 陷入了这个问题： https://jira.mongodb.org/browse/JAVA-2590
摘自link：
an older version of mongodb-driver-core is being pulled in
请清理你的依赖并使用最新的。