GraphX 无法正常工作 Spark / Scala

Question

我正在尝试在 apache Spark/Scala 中创建一个 GraphX 对象，但由于某种原因它似乎无法正常工作。我已经附上示例输入文件的文件，实际程序代码是：

package SGraph

import org.apache.spark._

import org.apache.spark.SparkContext._

import org.apache.spark.sql._

import org.apache.log4j._

import org.apache.spark.rdd.RDD

import org.apache.spark.graphx._
`
    object GooglePlusGraph {

  /** Our main function where the action happens */
      def main(args: Array[String]) {

    // Set the log level to only print errors
    Logger.getLogger("org").setLevel(Level.ERROR)

    // Create a SparkContext using every core of the local machine
    val sc = new SparkContext("local[*]", "GooglePlusGraphX") 

    val lines = sc.textFile("../Example.txt")

    val ratings = lines.map(x => x.toString().split(":")(0))

    val verts  = ratings.map(line => (line.toLong,line)) 

    val edges = lines.flatMap(makeEdges)  

    val default = "Nobody"
    val graph = Graph(verts, edges, default).cache()

    graph.degrees.join(verts).take(10).foreach(println)
    }

    def makeEdges(line: String) : List[Edge[Int]] = {

    import scala.collection.mutable.ListBuffer

    var edges = new ListBuffer[Edge[Int]]()

    val fields = line.split(",").flatMap(a => a.split(":"))

    val origin = fields(0)

    for (x <- 1 to (fields.length - 1)) {
      // Our attribute field is unused, but in other graphs could
      // be used to deep track of physical distances etc.
      edges += Edge(origin.toLong, fields(x).toLong, 0)
    }

    return edges.toList
  }
}

我得到的第一个错误如下：

16/12/19 01:28:33 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 3)
java.lang.NumberFormatException: For input string: "935750800736168978117"

感谢您的帮助！

Answer 1

跟你下面的问题是一样的问题

给定的号码有 21 位超出了最大位数 Long（19 位）。

GraphX 无法正常工作 Spark / Scala

GraphX not working properly Spark / Scala

scala

apache-spark

spark-graphx