ModuleNotFoundError: No module named 'sklearn.neighbors._dist_metrics'

Question

我训练了一个内核密度模型，然后使用 joblib 转储了该模型。然后我在调用相同的 .pkl 文件时创建了一个函数。它在我的本地机器上运行良好，但是当我将它部署到云机器上并使用它创建一个 docker 图像时，我收到以下错误之一：

ModuleNotFoundError: No module named 'sklearn.neighbors._dist_metrics'

或

ModuleNotFoundError: No module named 'sklearn.neighbors._kde'

可能导致此问题的原因是什么以及如何解决？

初始训练代码为：

import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import os
%matplotlib inline 
import seaborn as sns
import csv 
from sklearn.neighbors import KernelDensity
import joblib


arr = df_trim.values
kde = KernelDensity(kernel='gaussian', bandwidth=0.2).fit(arr)
joblib.dump(kde, 'kde.pkl')

 # This is the array that is used for training 
 # array([[3.5, 3.5, 3.5, 3.5],
       [4. , 4. , 3.5, 4. ],
       [3.5, 3. , 2.5, 3. ],
       ...,
       [2.5, 2.5, 2. , 2. ],
       [1.5, 1.5, 2. , 2.5],
       [3. , 3. , 2.5, 3. ]])

以下代码用于调用此已保存模型的函数：

from itertools import combinations
import joblib


filename = 'kde.pkl'  # filename for the model's pickle file.
model = joblib.load(filename) # loading the pre trained model using joblib.



def rSubset(arr, r):
  
    # return list of all subsets of length r
    # to deal with duplicate subsets use 
    # set(list(combinations(arr, r)))

    return list(combinations(arr, r))



def datapred(*args):
    
    no_args = len(args)
    args = list(args)
        
    pred_data = []
    model_score = []
    arr = [3.5 , 4 ,  3,  2.5,  1.5,  2,   1,   0.5,  0.25]
    n = (4 - no_args)
    comb_arr = (rSubset(arr, n))

    if(no_args==1):

        gpa1 = args[0]

        for i in range(1,len(comb_arr)):
                                      
            var = comb_arr[i]
            var = list(var)
            var = [gpa1]+var
            output = model.score_samples([var])
            model_score.append(output)
            pred_data.append(var)
            position = model_score.index(max(model_score))
            result = pred_data[position]
            return(result)

    elif(no_args==2):

        gpa1 = args[0]
        gpa2 = args[1] 

        for i in range(1,len(comb_arr)):
                                      
            var = comb_arr[i]
            var = list(var)
            var = [gpa1]+[gpa2]+var

            output = model.score_samples([var])
            model_score.append(output)
            pred_data.append(var)
            position = model_score.index(max(model_score))
            result = pred_data[position]
            return(result)

    elif(no_args==3):

        gpa1 = args[0]
        gpa2 = args[1]
        gpa3 = args[2] 

        for i in range(1,len(comb_arr)):
                                      
            var = comb_arr[i]
            var = list(var)
            var = [gpa1]+[gpa2]+[gpa3]+var

            output = model.score_samples([var])
            model_score.append(output)
            pred_data.append(var)
            position = model_score.index(max(model_score))
            result = pred_data[position]
            return(result)

此外，以下是 docker 图片的 requirements.txt 文件：

logger
Flask==1.1.2
Flask-RESTful==0.3.8
joblib==0.15.1
MarkupSafe==1.1.1
pandas==1.0.3
scikit-learn==0.19
sklearn >= 0.0
threadpoolctl==2.0.0
gunicorn==20.0.4
xgboost ==1.5.2
scipy >= 0.0

Answer 1

scikit-learn 库是您云计算机上的不同版本。

具体来说，sklearn.neighbors._dist_metrics 已在版本 1.0.2 附近被删除。也许您的 docker 容器实际上没有正确使用您的 requirements.txt。

以下是不同版本的示例：

这个不会抛出错误

>>> import sklearn
>>> sklearn.__version__
'0.24.2'
>>> from sklearn.neighbors import _dist_metrics

这个报错

>>> import sklearn
>>> sklearn.__version__
'1.0.2'
>>> from sklearn.neighbors import _dist_metrics
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name '_dist_metrics' from 'sklearn.neighbors'

ModuleNotFoundError: No module named 'sklearn.neighbors._dist_metrics'

ModuleNotFoundError: No module named 'sklearn.neighbors._dist_metrics'

python

machine-learning

scikit-learn

docker