sklearn grid.fit(X,y) - error: “positional indexers are out-of-bounds” for X_train,y_train
sklearn grid.fit(X,y) - error: “positional indexers are out-of-bounds” for X_train,y_train
这是关于 Python 2.7 和 Pandas 0.17.1 中的 scikit learn（版本 0.17.0）的问题。为了使用详细 的方法拆分原始数据（没有丢失的条目），我发现如果使用拆分数据继续进行 .fit()，则会出现错误。
这里的代码与另一个 Whosebug 问题基本没有变化，只是重命名了变量。然后我实例化了一个网格并尝试拟合拆分数据以确定最佳分类器参数。错误发生在以下代码的最后一行之后：
import pandas as pd
import numpy as np
# UCI's wine dataset
wine = pd.read_csv("https://s3.amazonaws.com/demo-datasets/wine.csv")
# separate target variable from dataset
y = wine['quality']
X = wine.drop(['quality','color'],axis = 1)
# Stratified Split of train and test data
from sklearn.cross_validation import StratifiedShuffleSplit
sss = StratifiedShuffleSplit(y, n_iter=3, test_size=0.2)
# Split dataset to obtain indices for train and test set
for train_index, test_index in sss:
xtrain, xtest = X.iloc[train_index], X.iloc[test_index]
ytrain, ytest = y[train_index], y[test_index]
# Pick some classifier here
from sklearn.tree import DecisionTreeClassifier
decision_tree = DecisionTreeClassifier()
from sklearn.grid_search import GridSearchCV
# Instantiate grid
grid = GridSearchCV(decision_tree, param_grid={'max_depth':np.arange(1,3)}, cv=sss, scoring='accuracy')
# this line causes the error message
grid.fit(xtrain,ytrain)
以上代码产生的错误信息如下：
Traceback (most recent call last):
File "C:\Python27\test.py", line 23, in <module>
grid.fit(xtrain,ytrain)
File "C:\Python27\lib\site-packages\sklearn\grid_search.py", line 804, in fit
return self._fit(X, y, ParameterGrid(self.param_grid))
File "C:\Python27\lib\site-packages\sklearn\grid_search.py", line 553, in _fit
for parameters in parameter_iterable
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 800, in __call__
while self.dispatch_one_batch(iterator):
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 658, in dispatch_one_batch
self._dispatch(tasks)
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 566, in _dispatch
job = ImmediateComputeBatch(batch)
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 180, in __init__
self.results = batch()
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 72, in __call__
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "C:\Python27\lib\site-packages\sklearn\cross_validation.py", line 1524, in _fit_and_score
X_train, y_train = _safe_split(estimator, X, y, train)
File "C:\Python27\lib\site-packages\sklearn\cross_validation.py", line 1591, in _safe_split
X_subset = safe_indexing(X, indices)
File "C:\Python27\lib\site-packages\sklearn\utils\__init__.py", line 152, in safe_indexing
return X.iloc[indices]
File "C:\Python27\lib\site-packages\pandas\core\indexing.py", line 1227, in __getitem__
return self._getitem_axis(key, axis=0)
File "C:\Python27\lib\site-packages\pandas\core\indexing.py", line 1504, in _getitem_axis
self._is_valid_list_like(key, axis)
File "C:\Python27\lib\site-packages\pandas\core\indexing.py", line 1443, in _is_valid_list_like
raise IndexError("positional indexers are out-of-bounds")
IndexError: positional indexers are out-of-bounds
注意：
将 X 和 y 保留为 Pandas 数据结构对我来说很重要，类似于上面另一个 Whosebug 问题中提出的第二种方法。即我不想使用 X.values 和 y.values.
问题：
使用原始数据作为 Pandas 数据结构（DataFrame 代表 X，Series 代表 y），有没有办法 运行 grid.fit() 没有收到此错误消息？
您应该将 X 和 y 直接传递给 fit()，例如
grid.fit(X, y)
和 GridSearchCV 将负责
xtrain, xtest = X.iloc[train_index], X.iloc[test_index]
ytrain, ytest = y[train_index], y[test_index]
StratifiedShuffleSplit 实例在迭代时产生成对的 train/test 拆分 索引 :
>>> list(sss)
[(array([2531, 4996, 4998, ..., 3205, 2717, 4983]), array([5942, 893, 1702, ..., 6340, 4806, 2537])),
(array([1888, 2332, 6276, ..., 1674, 775, 3705]), array([3404, 3304, 4741, ..., 4397, 3646, 1410])),
(array([1517, 3759, 4402, ..., 5098, 4619, 4521]), array([1110, 4076, 1280, ..., 6384, 1294, 1132]))]
GridSearchCV 将使用这些索引来分割训练样本。无需您手动操作。
错误发生是因为您将 xtrain 和 ytrain（train/test 拆分之一）输入交叉验证器。交叉验证器尝试访问存在于完整数据集中但不存在于 train/test 拆分中的项目，这会引发 IndexError.
这是关于 Python 2.7 和 Pandas 0.17.1 中的 scikit learn（版本 0.17.0）的问题。为了使用详细 .fit()，则会出现错误。
这里的代码与另一个 Whosebug 问题基本没有变化，只是重命名了变量。然后我实例化了一个网格并尝试拟合拆分数据以确定最佳分类器参数。错误发生在以下代码的最后一行之后：
import pandas as pd
import numpy as np
# UCI's wine dataset
wine = pd.read_csv("https://s3.amazonaws.com/demo-datasets/wine.csv")
# separate target variable from dataset
y = wine['quality']
X = wine.drop(['quality','color'],axis = 1)
# Stratified Split of train and test data
from sklearn.cross_validation import StratifiedShuffleSplit
sss = StratifiedShuffleSplit(y, n_iter=3, test_size=0.2)
# Split dataset to obtain indices for train and test set
for train_index, test_index in sss:
xtrain, xtest = X.iloc[train_index], X.iloc[test_index]
ytrain, ytest = y[train_index], y[test_index]
# Pick some classifier here
from sklearn.tree import DecisionTreeClassifier
decision_tree = DecisionTreeClassifier()
from sklearn.grid_search import GridSearchCV
# Instantiate grid
grid = GridSearchCV(decision_tree, param_grid={'max_depth':np.arange(1,3)}, cv=sss, scoring='accuracy')
# this line causes the error message
grid.fit(xtrain,ytrain)
以上代码产生的错误信息如下：
Traceback (most recent call last):
File "C:\Python27\test.py", line 23, in <module>
grid.fit(xtrain,ytrain)
File "C:\Python27\lib\site-packages\sklearn\grid_search.py", line 804, in fit
return self._fit(X, y, ParameterGrid(self.param_grid))
File "C:\Python27\lib\site-packages\sklearn\grid_search.py", line 553, in _fit
for parameters in parameter_iterable
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 800, in __call__
while self.dispatch_one_batch(iterator):
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 658, in dispatch_one_batch
self._dispatch(tasks)
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 566, in _dispatch
job = ImmediateComputeBatch(batch)
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 180, in __init__
self.results = batch()
File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 72, in __call__
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "C:\Python27\lib\site-packages\sklearn\cross_validation.py", line 1524, in _fit_and_score
X_train, y_train = _safe_split(estimator, X, y, train)
File "C:\Python27\lib\site-packages\sklearn\cross_validation.py", line 1591, in _safe_split
X_subset = safe_indexing(X, indices)
File "C:\Python27\lib\site-packages\sklearn\utils\__init__.py", line 152, in safe_indexing
return X.iloc[indices]
File "C:\Python27\lib\site-packages\pandas\core\indexing.py", line 1227, in __getitem__
return self._getitem_axis(key, axis=0)
File "C:\Python27\lib\site-packages\pandas\core\indexing.py", line 1504, in _getitem_axis
self._is_valid_list_like(key, axis)
File "C:\Python27\lib\site-packages\pandas\core\indexing.py", line 1443, in _is_valid_list_like
raise IndexError("positional indexers are out-of-bounds")
IndexError: positional indexers are out-of-bounds
注意：
将 X 和 y 保留为 Pandas 数据结构对我来说很重要，类似于上面另一个 Whosebug 问题中提出的第二种方法。即我不想使用 X.values 和 y.values.
问题：
使用原始数据作为 Pandas 数据结构（DataFrame 代表 X，Series 代表 y），有没有办法 运行 grid.fit() 没有收到此错误消息？
您应该将 X 和 y 直接传递给 fit()，例如
grid.fit(X, y)
和 GridSearchCV 将负责
xtrain, xtest = X.iloc[train_index], X.iloc[test_index]
ytrain, ytest = y[train_index], y[test_index]
StratifiedShuffleSplit 实例在迭代时产生成对的 train/test 拆分 索引 :
>>> list(sss)
[(array([2531, 4996, 4998, ..., 3205, 2717, 4983]), array([5942, 893, 1702, ..., 6340, 4806, 2537])),
(array([1888, 2332, 6276, ..., 1674, 775, 3705]), array([3404, 3304, 4741, ..., 4397, 3646, 1410])),
(array([1517, 3759, 4402, ..., 5098, 4619, 4521]), array([1110, 4076, 1280, ..., 6384, 1294, 1132]))]
GridSearchCV 将使用这些索引来分割训练样本。无需您手动操作。
错误发生是因为您将 xtrain 和 ytrain（train/test 拆分之一）输入交叉验证器。交叉验证器尝试访问存在于完整数据集中但不存在于 train/test 拆分中的项目，这会引发 IndexError.