在 GraphX 中从 RDD[Long, Long, ED] 构造一个 RDD[Edge[ED]]

Question

我现在正在尝试从 RDD[(Long,Long,String)] 构造一个 RDD[Edge[String]] 我看过Edge的src代码，它有Edge(VertexId,VertexId,ED)的参数，所以我想如果我有一个RDD[(Long,Long,String)]我可以把它转换成RDD[Edge[String]]

我的密码是

val c:RDD[(String, String)] = something
val s:RDD[(String, String)] = something
val edgeRDD: RDD[Edge[String]] = c
        .join(s).map({
            case (num: String, (src: String, dst: String)) 
                => (src.toLong, dst.toLong, expRelation)
        }
    )

但最后我得到了一个type mismatch错误

[error]  found   : (Long, Long, String)
[error]  required: org.apache.spark.graphx.Edge[String]
[error]                     => (src.toLong, dst.toLong, expRelation)
[error]                        ^

所以我想知道是否有一种方法可以直接从RDD[(Long,Long,ED)]

构造一个RDD[Edge[ED]]

Answer 1

您的函数不是 returning Edge[String]。将 return 类型更改为

Edge(src.toLong, dst.toLong, expRelation)

代码：

val c:RDD[(String, String)] = something
val s:RDD[(String, String)] = something
val edgeRDD: RDD[Edge[String]] = c
        .join(s).map({
            case (num: String, (src: String, dst: String)) 
                => Edge(src.toLong, dst.toLong, expRelation)
        }
    )

在 GraphX 中从 RDD[Long, Long, ED] 构造一个 RDD[Edge[ED]]

Construct a RDD[Edge[ED]] from RDD[Long, Long, ED] in GraphX

scala

apache-spark

rdd

spark-graphx