将 Rdd 转换为列表

Question

如何将此 Rdd 转换为列表：

tfidf.collect()
output:
[DenseVector([0.0619]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.0619]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031])]

而且我需要将此 Rdd 设为与此类似的列表：

tfidf_list= [
    DenseVector([0.0619]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.0619]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031]),
 DenseVector([0.031])
]

感谢您的帮助。

Answer 1

使用这个：

tfidf_list = tfidf.collect()

将 Rdd 转换为列表

Convert Rdd to list

python

pyspark

jupyter-notebook