规模 - Java.lang.UnsupportedOperationException

Question

我得到

java.lang.UnsupportedOperationException: Schema for type Range.Value is not supported.

感谢任何对此的指点

object Range extends Enumeration {
  type Range = Value

  val RangeMedium = Value("Range Medium")
  val RangeHigh = Value("Range Higher")
  val RangeNotEnough = Value("Range Not enough")
  val NotApplicable = Value("Not Applicable")

}



val getRange = udf((p1: Double, p2: Double) => {
    if (p1 >= 5 && p1 < 10 && p2 >= 1) {
      Some(Range.RangeMedium)
    }
    else if (p1 >= 10 && p2 >= 1) {
      Some(Range.RangeHigh)
    }
    else {
      Some(Range.NotApplicable)
    }
  })

ds = Seq(9,10).toDF("p1","p2")

ds.withColumn("level",getRange($"p1",$"p2")).show()

Answer 1

如果您要从 UDF 返回字符串，您可以尝试使用 .toString:

将枚举值转换为字符串

object Range extends Enumeration {
  type Range = Value

  val RangeMedium = Value("Range Medium")
  val RangeHigh = Value("Range Higher")
  val RangeNotEnough = Value("Range Not enough")
  val NotApplicable = Value("Not Applicable")
}

val getRange = udf((p1: Double, p2: Double) => {
    if (p1 >= 5 && p1 < 10 && p2 >= 1) {
      Range.RangeMedium.toString
    }
    else if (p1 >= 10 && p2 >= 1) {
      Range.RangeHigh.toString
    }
    else {
      Range.NotApplicable.toString
    }
})

val ds = Seq((9,10)).toDF("p1","p2")

ds.withColumn("level",getRange($"p1",$"p2")).show()
+---+---+------------+
| p1| p2|       level|
+---+---+------------+
|  9| 10|Range Medium|
+---+---+------------+

话虽如此，这种操作也可以使用 when 语句而不是 UDF，这应该会更高效。

规模 - Java.lang.UnsupportedOperationException

Scala enum - Java.lang.UnsupportedOperationException

scala

user-defined-functions

apache-spark