Spark Scala - 通过 VertexID 连接两个数组
Spark Scala - Joining two arrays by VertexID
我有 2 个格式如下的数组
scala> cPV.take(5)
res18: Array[(org.apache.spark.graphx.VertexId, String)] = Array((-496366541,7804412), (183389035,11517829), (1300761459,36164965), (978932066,32135154), (370291237,40355685))
scala> fC.take(5)
res19: Array[(org.apache.spark.graphx.VertexId, Int)] = Array((386253628,1), (-1141923433,1), (1871855296,7), (1938255756,1), (-749015657,5))
我需要加入他们才能进入格式 - Array[(org.apache.spark.graphx.VertexId, Int, String)]
我试过 .join() 但它抛出以下错误
val mVP = fC.join(cPV)
<console>:64: error: value join is not a member of Array[(org.apache.spark.graphx.VertexId, Int)]
val mVP = fC.join(cPV)
我也试过但没用。
我尝试了以下方法并且有效
val fCRDD = sc.parallelize(fC)
scala> val mVP = fCRDD.join(cPV)
mVP: org.apache.spark.rdd.RDD[(org.apache.spark.graphx.VertexId, (Int, String))] = MapPartitionsRDD[106] at join at <console>:67
scala> mVP.take(5)
res21: Array[(org.apache.spark.graphx.VertexId, (Int, String))] = Array((-891966589,(4,D)), (166544732,(74,V)), (1871855296,(7,LG)), (1416009424,(6,Dck)), (-241988197,(4,L)))
抱歉,这里是新手 - 我应该在此处发布问题之前先尝试一下。
我有 2 个格式如下的数组
scala> cPV.take(5)
res18: Array[(org.apache.spark.graphx.VertexId, String)] = Array((-496366541,7804412), (183389035,11517829), (1300761459,36164965), (978932066,32135154), (370291237,40355685))
scala> fC.take(5)
res19: Array[(org.apache.spark.graphx.VertexId, Int)] = Array((386253628,1), (-1141923433,1), (1871855296,7), (1938255756,1), (-749015657,5))
我需要加入他们才能进入格式 - Array[(org.apache.spark.graphx.VertexId, Int, String)]
我试过 .join() 但它抛出以下错误
val mVP = fC.join(cPV)
<console>:64: error: value join is not a member of Array[(org.apache.spark.graphx.VertexId, Int)]
val mVP = fC.join(cPV)
我也试过
我尝试了以下方法并且有效
val fCRDD = sc.parallelize(fC)
scala> val mVP = fCRDD.join(cPV)
mVP: org.apache.spark.rdd.RDD[(org.apache.spark.graphx.VertexId, (Int, String))] = MapPartitionsRDD[106] at join at <console>:67
scala> mVP.take(5)
res21: Array[(org.apache.spark.graphx.VertexId, (Int, String))] = Array((-891966589,(4,D)), (166544732,(74,V)), (1871855296,(7,LG)), (1416009424,(6,Dck)), (-241988197,(4,L)))
抱歉,这里是新手 - 我应该在此处发布问题之前先尝试一下。