Scala

Question

我正在经历一些与我的理解不符的事情。我的理解是 'this' 对于活动对象不能为空，但是，对于下面显示的情况，我遇到了那种情况。

上下文 - 我在这种情况下使用 XGBoost4J-Spark 包。可以看看源码here. More specifically, I am referring to the XGBoostEstimatorclass。我有以下 class 的定义，只有一个额外的打印语句。

package ml.dmlc.xgboost4j.scala.spark

import ml.dmlc.xgboost4j.scala.{EvalTrait, ObjectiveTrait}
import org.apache.spark.ml.{Predictor, Estimator}
import org.apache.spark.ml.param.ParamMap
import org.apache.spark.ml.util.Identifiable
import org.apache.spark.mllib.linalg.{VectorUDT, Vector}
import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.sql.functions._
import org.apache.spark.sql.types.{NumericType, DoubleType, StructType}
import org.apache.spark.sql.{DataFrame, TypedColumn, Dataset, Row}

/**
 * the estimator wrapping XGBoost to produce a training model
 *
 * @param inputCol the name of input column
 * @param labelCol the name of label column
 * @param xgboostParams the parameters configuring XGBoost
 * @param round the number of iterations to train
 * @param nWorkers the total number of workers of xgboost
 * @param obj the customized objective function, default to be null and using the default in model
 * @param eval the customized eval function, default to be null and using the default in model
 * @param useExternalMemory whether to use external memory when training
 * @param missing the value taken as missing
 */
class XGBoostEstimator(
    inputCol: String, labelCol: String,
    xgboostParams: Map[String, Any], round: Int, nWorkers: Int,
    obj: Option[ObjectiveTrait] = None,
    eval: Option[EvalTrait] = None, useExternalMemory: Boolean = false, missing: Float = Float.NaN)
  extends Estimator[XGBoostModel] {

  println(s"This is ${this}")
  override val uid: String = Identifiable.randomUID("XGBoostEstimator")


  /**
   * produce a XGBoostModel by fitting the given dataset
   */
  def fit(trainingSet: Dataset[_]): XGBoostModel = {
    val instances = trainingSet.select(
      col(inputCol), col(labelCol).cast(DoubleType)).rdd.map {
      case Row(feature: Vector, label: Double) =>
        LabeledPoint(label, feature)
    }
    transformSchema(trainingSet.schema, logging = true)
    val trainedModel = XGBoost.trainWithRDD(instances, xgboostParams, round, nWorkers, obj.get,
      eval.get, useExternalMemory, missing).setParent(this)
    copyValues(trainedModel)
  }

  override def copy(extra: ParamMap): Estimator[XGBoostModel] = {
    defaultCopy(extra)
  }

  override def transformSchema(schema: StructType): StructType = {
    // check input type, for now we only support vectorUDT as the input feature type
    val inputType = schema(inputCol).dataType
    require(inputType.equals(new VectorUDT), s"the type of input column $inputCol has to VectorUDT")
    // check label Type,
    val labelType = schema(labelCol).dataType
    require(labelType.isInstanceOf[NumericType], s"the type of label column $labelCol has to" +
      s" be NumericType")
    schema
  }
}

当我通过 Spark-Shell（或以其他方式通过测试）初始化相同的代码时，以下是我得到的输出：

scala> import ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator
import ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator

scala> val xgb = new XGBoostEstimator("features", "label", Map.empty,10, 2)
This is null
xgb: ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator = XGBoostEstimator_6cd31d495c8f

scala> xgb.uid
res1: String = XGBoostEstimator_6cd31d495c8f

任何关于为什么以及何时可能出现此行为的说明都会有所帮助。

Answer 1

您的 toString() 实现来自 Identifiable，它只是 returns uid 集。并且由于您在下一行中设置了 uid，因此它在打印时未初始化。

可识别source:

trait Identifiable {

  /**
   * An immutable unique ID for the object and its derivatives.
   */
  val uid: String

  override def toString: String = uid
}

Scala - 'this' 在 Scala 中对于活动对象可以为 null 吗？

Scala - can 'this' be null in Scala for a live object?

apache-spark

xgboost