Cassandra连接器--joinWithCassandraTable和leftJoinWithCassandraTable的区别--无法解析符号

Question

我正在尝试通过使用 datastax cassandra 连接器加入来访问来自 Cassandra 的数据。下面的代码对我有用。我试图在 join

之后总结来自 RDD 和 Cassandra 的值列

tm(a.joinWithCassandraTable("ks","tbl").on(SomeColumns("key","key2","key3","key4","key5","key6","key7","key8","key9","key10","key11","key12","key13","key14","key15","column1","column2","column3","column4","column5")).select("value1").map { case (ip, row) => IP(ip.key, ip.key2, ip.key3,ip.key4,ip.key5,ip.key6,ip.key7,ip.key8,ip.key9,ip.key10,ip.key11,ip.key12,ip.key13,ip.key14,ip.key15,ip.column1,ip.column2,ip.column3,ip.column4,ip.column5,ip.value1 + row.getLong("value1")) }.saveToCassandra("ks", "tbl"))

但是，当我尝试进行左连接时，它给出了 "Cannot Resolve symbol getLong" 我相信这是因为左连接不能保证一个值，因为它可能为空，但我无法在 Scala 中对此进行编码。

tm(a.leftJoinWithCassandraTable("ks","tbl").on(SomeColumns("key","key2","key3","key4","key5","key6","key7","key8","key9","key10","key11","key12","key13","key14","key15","column1","column2","column3","column4","column5")).select("value1").map { case (ip, row) => IP(ip.key, ip.key2, ip.key3,ip.key4,ip.key5,ip.key6,ip.key7,ip.key8,ip.key9,ip.key10,ip.key11,ip.key12,ip.key13,ip.key14,ip.key15,ip.column1,ip.column2,ip.column3,ip.column4,ip.column5,ip.value1 + row.getLong("value1")) }.saveToCassandra("ks", "tbl"))

感谢任何帮助。如果有任何需要的信息，请告诉我，我会尽力添加

Answer 1

当您没有在 Cassandra 中获取数据时，您应该获取一个 Option[Row] 而不是 Row 对象。

而不是 .map { case (ip, row) => ...} 你可以这样写：

.map { case (ip, row) => 
  row match {
    case None => ip
    case Some(data) => IP(...., ip.value1 + data.getLong("value1"))
  }
}

在这种情况下 - 当你没有数据时 (None)，那么你只是 return IP 对象本身，如果你有数据那么你构造新的 IP 对象

Cassandra连接器--joinWithCassandraTable和leftJoinWithCassandraTable的区别--无法解析符号

Cassandra connector -- difference between joinWithCassandraTable and leftJoinWithCassandraTable -- Cannot Resolve symbol

apache-spark

cassandra

datastax

spark-cassandra-connector