error: object xml is not a member of package com.databricks.spark

Question

我正在尝试使用 SBT 读取 XML 文件，但我在编译它时遇到问题。

build.sbt

name:= "First Spark"
version:= "1.0"
organization := "in.goai"
scalaVersion := "2.11.8"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.0.0"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.0.0"
libraryDependencies += "com.databricks" % "spark-avro_2.10" % "2.0.1"
libraryDependencies += "org.scala-lang.modules" %% "scala-xml" % "1.0.2"
resolvers += Resolver.mavenLocal

.scala 文件

package in.goai.spark

import scala.xml._
import com.databricks.spark.xml
import org.apache.spark.sql.SQLContext
import org.apache.spark.{SparkContext, SparkConf}

object SparkMeApp {
  def main(args: Array[String]) {
    val conf = new SparkConf().setAppName("First Spark")
    val sc = new SparkContext(conf)
    val sqlContext = new SQLContext(sc)
    val fileName = args(0)
    val df = sqlContext.read.format("com.databricks.spark.xml").option("rowTag", "book").load("fileName")
    val selectedData = df.select("title", "price")
    val d = selectedData.show
    println(s"$d")

  }
}

当我通过给定 "sbt package" 编译它时，它显示以下错误

[error] /home/hadoop/dev/first/src/main/scala/SparkMeApp.scala:4: object xml is not a member of package com.databricks.spark
[error] import com.databricks.spark.xml
[error]        ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 9 s, completed Sep 22, 2017 4:11:19 PM

我是否需要添加任何其他与 xml 相关的 jar 文件？请提出建议并向我提供任何 link，它提供有关不同文件格式的 jar 文件的信息

Answer 1

因为您使用的是 Scala 2.11 和 Spark 2.0，所以在 build.sbt 中，将您的依赖项更改为以下内容：

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.0.0"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.0.0"
libraryDependencies += "com.databricks" %% "spark-avro" % "3.2.0"
libraryDependencies += "com.databricks" %% "spark-xml" % "0.4.1"
libraryDependencies += "org.scala-lang.modules" %% "scala-xml" % "1.0.6"

将 spark-avro 版本更改为 3.2.0：https://github.com/databricks/spark-avro#requirements
添加"com.databricks" %% "spark-xml" % "0.4.1"：https://github.com/databricks/spark-xml#scala-211
将 scala-xml 版本更改为 1.0.6，Scala 2.11 的当前版本：http://mvnrepository.com/artifact/org.scala-lang.modules/scala-xml_2.11

在您的代码中，删除以下导入语句：

import com.databricks.spark.xml

请注意，您的代码实际上并未使用 spark-avro 或 scala-xml 库。如果您不打算使用它们，请从 build.sbt（以及代码中的 import scala.xml._ 语句）中删除这些依赖项。

error: object xml is not a member of package com.databricks.spark

error: object xml is not a member of package com.databricks.spark

scala

apache-spark

sbt

apache-spark-sql

scala-xml