如何在 Spark (Scala) 中恢复 One-Hot 编码
How to revert One-Hot Enoding in Spark (Scala)
在 运行 k-means (mllib spark scala) 之后，我想弄清楚我从使用 mllib 的 OneHotEncoder（以及其他转换器）预处理的数据中获得的聚类中心。
中心看起来像这样：
Cluster Center 0 [0.3496378699559276,0.05482645034473324,111.6962521358467,1.770525792286651,0.0,0.8561916265130964,0.014382183950365071,0.0,0.0,0.0,0.47699722692567864,0.0,0.0,0.0,0.04988557988346689,0.0,0.0,0.0,0.8981811028926263,0.9695107580117296,0.0 ,0.0,1.7505886931570156,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 ,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 ,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 ,0.0,0.0,0.0,0.0,0.0,0.0,17.771620072281845,0.0,0.0,0.0,0.0]
这显然不是很人性化...关于如何恢复单热编码和检索原始分类特征的任何想法？
如果我寻找最接近质心的数据点（使用 k-means 使用的相同距离度量，我假设它是欧几里德距离）然后恢复该特定数据点的编码怎么办？
对于簇质心，不可能（强烈不推荐）反转编码。想象一下，你有 6 个中的原始特征“3”，它被编码为 [0.0,0.0,1.0,0.0,0.0,0.0]。在这种情况下，很容易从编码中提取 3 作为正确的特征。
但是在应用 kmeans 之后，您可能会得到一个像这样寻找此功能的集群质心 [0.0,0.13,0.0,0.77,0.1,0.0]。如果您想将其解码回您之前的表示，例如 6 中的“4”，因为特征 4 具有最大值，那么您将丢失信息并且模型可能会损坏。
编辑：添加一种可能的方法来将数据点的编码从评论恢复为答案
如果您在数据点上有 ID，则可以在将数据点分配给集群后，在编码之前对 ID 执行 select/join 操作以获取旧状态。
在 运行 k-means (mllib spark scala) 之后，我想弄清楚我从使用 mllib 的 OneHotEncoder（以及其他转换器）预处理的数据中获得的聚类中心。
中心看起来像这样：
Cluster Center 0 [0.3496378699559276,0.05482645034473324,111.6962521358467,1.770525792286651,0.0,0.8561916265130964,0.014382183950365071,0.0,0.0,0.0,0.47699722692567864,0.0,0.0,0.0,0.04988557988346689,0.0,0.0,0.0,0.8981811028926263,0.9695107580117296,0.0 ,0.0,1.7505886931570156,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 ,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 ,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 ,0.0,0.0,0.0,0.0,0.0,0.0,17.771620072281845,0.0,0.0,0.0,0.0]
这显然不是很人性化...关于如何恢复单热编码和检索原始分类特征的任何想法？ 如果我寻找最接近质心的数据点（使用 k-means 使用的相同距离度量，我假设它是欧几里德距离）然后恢复该特定数据点的编码怎么办？
对于簇质心，不可能（强烈不推荐）反转编码。想象一下，你有 6 个中的原始特征“3”，它被编码为 [0.0,0.0,1.0,0.0,0.0,0.0]。在这种情况下，很容易从编码中提取 3 作为正确的特征。
但是在应用 kmeans 之后，您可能会得到一个像这样寻找此功能的集群质心 [0.0,0.13,0.0,0.77,0.1,0.0]。如果您想将其解码回您之前的表示，例如 6 中的“4”，因为特征 4 具有最大值，那么您将丢失信息并且模型可能会损坏。
编辑：添加一种可能的方法来将数据点的编码从评论恢复为答案
如果您在数据点上有 ID，则可以在将数据点分配给集群后，在编码之前对 ID 执行 select/join 操作以获取旧状态。