Fit Error OneVsOneClassifier, OneVsRestClassifier CPU READ on GPU P100 for digits recognition mulit-label classifier fit model

Question

问题

目标是使用 RandomisedSearchCV 调整参数，然后拟合两个模型，OneVsOneClassifier 和 OneVsRestClassifier 各一个。然后使用来自 RandomizedSearchCV 的调整参数检查每个模型的准确性性能。定义两个模型来拟合，以拟合数字识别 MNIST 数据集以进行多标签分类预测。

我使用 GridSearchCV 设置了一个调整超参数，对于 estimator__C' 来说很简单：[0.1, 1, 100, 200] 对于 LogisticRegression。为了审核，我打印了计算出的网格参数。为拟合模型提供缩放的 X-train 对象。然后运行拟合模型。

问题在 Kaggle GPU P100 上运行。当我执行代码：ovr_grid_search.fit() & ovo_grid_search.fit() 时，一切都在无限运行。

适配错误（详细=1）

FitFailedWarning: 
50 fits failed out of a total of 50.
The score on these train-test partitions for these parameters will be set to nan.
If these failures are not expected, you can try to debug them by setting error_score='raise'.

Below are more details about the failures:
--------------------------------------------------------------------------------
50 fits failed with the following error:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 680, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
TypeError: fit() got an unexpected keyword argument 'verbose'

  warnings.warn(some_fits_failed_message, FitFailedWarning)
/opt/conda/lib/python3.7/site-packages/sklearn/model_selection/_search.py:972: UserWarning: One or more of the test scores are non-finite: [nan nan nan nan nan nan nan nan nan nan]
  category=UserWarning,
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_392/86354045.py in <module>
      1 # Use the best models to make predictions and evaluate performance
----> 2 ovr_grid_param.fit(X_train_scaled, y_train, verbose = 1)
      3 ovr_grid_param.fit(X_train_scaled, y_train, verbose = 1)

/opt/conda/lib/python3.7/site-packages/sklearn/model_selection/_search.py in fit(self, X, y, groups, **fit_params)
    924             refit_start_time = time.time()
    925             if y is not None:
--> 926                 self.best_estimator_.fit(X, y, **fit_params)
    927             else:
    928                 self.best_estimator_.fit(X, **fit_params)

代码

from sklearn.linear_model import LogisticRegression
from sklearn.multiclass import OneVsOneClassifier, OneVsRestClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train.astype(np.float64))

ovr_model = OneVsRestClassifier(LogisticRegression())
ovo_model = OneVsOneClassifier(LogisticRegression())

param_grid = {
    'estimator__C': [0.1, 1, 100, 200]
}

ovr_grid_param = RandomizedSearchCV(ovr_model, param_grid, cv=5, n_jobs=8)
ovo_grid_param = RandomizedSearchCV(ovo_model, param_grid, cv=5, n_jobs=8)

print("OneVsRestClassifier best params: ", ovr_grid_param)
print("OneVsOneClassifier best params: ", ovo_grid_param)


### below code is the problem area **

ovr_grid_param.fit(X_train_scaled, y_train)
ovo_grid_param.fit(X_train_scaled, y_train)

数据 数字识别 MNIST 数据集。 X_train 缩放数据

array([[0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
       ...,
       [0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.]])

y_train 数据

33092    5
30563    2
17064    4
16679    9
30712    0
30177    0
11735    3
1785     8
4382     3
21702    7
37516    3
9476     6
4893     5
22117    0
12646    8

GridSearch 执行结果

OneVsRestClassifier best params:  GridSearchCV(cv=5, error_score='raise',
       estimator=OneVsRestClassifier(estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
          verbose=0, warm_start=False),
          n_jobs=1),
       fit_params=None, iid=True, n_jobs=-1,
       param_grid={'estimator__C': [0.1, 1, 100, 200]},
       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',
       scoring=None, verbose=0)
OneVsOneClassifier best params:  GridSearchCV(cv=5, error_score='raise',
       estimator=OneVsOneClassifier(estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
          verbose=0, warm_start=False),
          n_jobs=1),
       fit_params=None, iid=True, n_jobs=-1,
       param_grid={'estimator__C': [0.1, 1, 100, 200]},
       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',
       scoring=None, verbose=0)

Answer 1

我不知道 sklearn GPU 支持 https://scikit-learn.org/stable/faq.html#will-you-add-gpu-support

有没有办法运行“nvidia-smi”？您可以检查 P100 上是否有活动的进程。否则，可能只是因为网格搜索空间太大而花费的时间太长。尝试增加 n_jobs 参数以利用更多 cpu。