如何在 Apache Spark 2 中设置广播工厂方法
How to setup broadcast factory method in Apache Spark 2
在 Apache Spark 1.X 中,有 a configuration setting named: "spark.broadcast.factory"
where you could choose a broadcast factory. That setting, seems to be no longer supported in Apache Spark 2.X (it's not in the configurations list)。某人如何控制 Apache Spark 2.X 中的广播工厂?默认的广播工厂是什么?以前是TorrentBroadcastFactory
.
此设置已通过此提交从 2.X 中删除:
https://github.com/apache/spark/commit/ee8f8d318417c514fbb26e57157483d466ddbfae#diff-c3302615ef6dce00341e156431369d38
有关原因的更多详细信息,请参阅此 JIRA:https://issues.apache.org/jira/browse/SPARK-12588 and https://issues.apache.org/jira/browse/SPARK-18742
编辑:spark.broadcast.factory
默认设置为 TorrentBroadcastFactory
在 Apache Spark 1.X 中,有 a configuration setting named: "spark.broadcast.factory"
where you could choose a broadcast factory. That setting, seems to be no longer supported in Apache Spark 2.X (it's not in the configurations list)。某人如何控制 Apache Spark 2.X 中的广播工厂?默认的广播工厂是什么?以前是TorrentBroadcastFactory
.
此设置已通过此提交从 2.X 中删除: https://github.com/apache/spark/commit/ee8f8d318417c514fbb26e57157483d466ddbfae#diff-c3302615ef6dce00341e156431369d38
有关原因的更多详细信息,请参阅此 JIRA:https://issues.apache.org/jira/browse/SPARK-12588 and https://issues.apache.org/jira/browse/SPARK-18742
编辑:spark.broadcast.factory
默认设置为 TorrentBroadcastFactory