java.lang.ClassNotFoundException：找不到数据源：org.apache.bahir.sql.streaming.mqtt.MQTTStreamSinkProvider。请找包裹

Question

我已经添加了

"org.apache.bahir" %% "spark-streaming-mqtt" % "2.4.0"

到我的build.sbt，并使用

df
  .writeStream
  .format("org.apache.bahir.sql.streaming.mqtt.MQTTStreamSinkProvider")
  .outputMode("complete")
  .option("topic", "mytopic")
  .option("brokerUrl", "tcp://localhost:1883")
  .start()
  .awaitTermination(20000)

在代码中，但得到

java.lang.ClassNotFoundException: Failed to find data source: org.apache.bahir.sql.streaming.mqtt.MQTTStreamSinkProvider. Please find packages at http://spark.apache.org/third-party-projects.html

    at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:679)

Answer 1

我已经意识到这种行为的原因是使用了错误的依赖项 "org.apache.bahir" %% "spark-streaming-mqtt" % "2.4.0" 而不是 "org.apache.bahir" %% "spark-sql-streaming-mqtt" % "2.4.0"

修复后又遇到新问题

java.lang.ClassNotFoundException：找不到数据源：org.apache.bahir.sql.streaming.mqtt.MQTTStreamSinkProvider。请找包裹

java.lang.ClassNotFoundException: Failed to find data source: org.apache.bahir.sql.streaming.mqtt.MQTTStreamSinkProvider. Please find packages

scala

mqtt

apache-spark

spark-streaming

apache-bahir