扩展 xgboost.XGBClassifier

Question

我正在尝试定义一个名为 XGBExtended 的 class，它扩展了 class xgboost.XGBClassifier，xgboost 的 scikit-learn API。我运行遇到了 get_params 方法的一些问题。下面是一个 IPython 会话来说明这个问题。基本上，get_params 似乎只返回我在 XGBExtended.__init__ 中定义的属性，而在父初始化方法 (xgboost.XGBClassifier.__init__) 期间定义的属性将被忽略。我正在使用 IPython 和运行 python 2.7。底部的完整系统规格。

In [182]: import xgboost as xgb
     ...: 
     ...: class XGBExtended(xgb.XGBClassifier):
     ...:   def __init__(self, foo):
     ...:     super(XGBExtended, self).__init__()
     ...:     self.foo = foo
     ...: 
     ...: clf = XGBExtended(foo = 1)
     ...: 
     ...: clf.get_params()
     ...: 
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-182-431c4c3f334b> in <module>()
      8 clf = XGBExtended(foo = 1)
      9 
---> 10 clf.get_params()

/Users/andrewhannigan/lib/xgboost/python-package/xgboost/sklearn.pyc in get_params(self, deep)
    188         if isinstance(self.kwargs, dict):  # if kwargs is a dict, update params accordingly
    189             params.update(self.kwargs)
--> 190         if params['missing'] is np.nan:
    191             params['missing'] = None  # sklearn doesn't handle nan. see #4725
    192         if not params.get('eval_metric', True):

KeyError: 'missing'

所以我遇到了一个错误，因为 'missing' 不是 XGBClassifier.get_params 方法中 params 字典中的键。我进入调试器四处寻找：

In [183]: %debug
> /Users/andrewhannigan/lib/xgboost/python-package/xgboost/sklearn.py(190)get_params()
    188         if isinstance(self.kwargs, dict):  # if kwargs is a dict, update params accordingly
    189             params.update(self.kwargs)
--> 190         if params['missing'] is np.nan:
    191             params['missing'] = None  # sklearn doesn't handle nan. see #4725
    192         if not params.get('eval_metric', True):

ipdb> params
{'foo': 1}
ipdb> self.__dict__
{'n_jobs': 1, 'seed': None, 'silent': True, 'missing': nan, 'nthread': None, 'min_child_weight': 1, 'random_state': 0, 'kwargs': {}, 'objective': 'binary:logistic', 'foo': 1, 'max_depth': 3, 'reg_alpha': 0, 'colsample_bylevel': 1, 'scale_pos_weight': 1, '_Booster': None, 'learning_rate': 0.1, 'max_delta_step': 0, 'base_score': 0.5, 'n_estimators': 100, 'booster': 'gbtree', 'colsample_bytree': 1, 'subsample': 1, 'reg_lambda': 1, 'gamma': 0}
ipdb>

如您所见，params 仅包含 foo 变量。但是，对象本身包含 xgboost.XGBClassifier.__init__ 定义的所有参数。但出于某种原因，从 xgboost.XGBClassifier.get_params 调用的 BaseEstimator.get_params 方法仅获取在 XGBExtended.__init__ 方法中明确定义的参数。不幸的是，即使我用 deep = True 显式调用 get_params，它仍然无法正常工作：

ipdb> super(XGBModel, self).get_params(deep=True)
{'foo': 1}
ipdb>

谁能告诉我为什么会这样？

系统规格：

In [186]: print IPython.sys_info()
{'commit_hash': u'1149d1700',
 'commit_source': 'installation',
 'default_encoding': 'UTF-8',
 'ipython_path': '/Users/andrewhannigan/virtualenvironment/nimble_ai/lib/python2.7/site-packages/IPython',
 'ipython_version': '5.4.1',
 'os_name': 'posix',
 'platform': 'Darwin-14.5.0-x86_64-i386-64bit',
 'sys_executable': '/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python',
 'sys_platform': 'darwin',
 'sys_version': '2.7.10 (default, Jul  3 2015, 12:05:53) \n[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)]'}

Answer 1

这里的问题是 child class 的声明不正确。当您仅使用 foo 声明 init 方法时，您将覆盖原始方法。它不会自动初始化，即使基本 class 构造函数应该为它们设置默认值。

您应该使用以下内容：

class XGBExtended(xgb.XGBClassifier):
    def __init__(self, foo, max_depth=3, learning_rate=0.1,
                 n_estimators=100, silent=True,
                 objective="binary:logistic",
                 nthread=-1, gamma=0, min_child_weight=1,
                 max_delta_step=0, subsample=1, colsample_bytree=1, colsample_bylevel=1,
                 reg_alpha=0, reg_lambda=1, scale_pos_weight=1,
                 base_score=0.5, seed=0, missing=None, **kwargs):

        # Pass the required parameters to super class
        super(XGBExtended, self).__init__(max_depth, learning_rate,
                                            n_estimators, silent, objective,
                                            nthread, gamma, min_child_weight,
                                            max_delta_step, subsample,
                                            colsample_bytree, colsample_bylevel,
                                            reg_alpha, reg_lambda,
scale_pos_weight, base_score, seed, missing, **kwargs)

        # Use other custom parameters
        self.foo = foo

之后你将不会收到任何错误。

clf = XGBExtended(foo = 1)
print(clf.get_params(deep=True))

>>> {'reg_alpha': 0, 'colsample_bytree': 1, 'silent': True, 
     'colsample_bylevel': 1, 'scale_pos_weight': 1, 'learning_rate': 0.1, 
     'missing': None, 'max_delta_step': 0, 'nthread': -1, 'base_score': 0.5, 
     'n_estimators': 100, 'subsample': 1, 'reg_lambda': 1, 'seed': 0, 
     'min_child_weight': 1, 'objective': 'binary:logistic', 
     'foo': 1, 'max_depth': 3, 'gamma': 0}

扩展 xgboost.XGBClassifier

Extending xgboost.XGBClassifier

python

python-2.7

scikit-learn

xgboost