加载保存的文件时 Flask 出错并显示 dill.dill 消息

Question

我在 s3 中保存了两个文件（模型、矢量化器）并尝试从 s3 下载并运行它与我的烧瓶应用程序文件。但是，目前 returns dill.dill（没有模块命名错误）。有趣的是，我没有在任何地方导入 "dill" 但它看起来像是在内部导入并抱怨。

    import flask
    from flask import Flask, request, render_template, jsonify
    from sklearn.externals import joblib
    import numpy as np
    from scipy import misc
    from sklearn.feature_extraction.text import TfidfVectorizer
    import boto3
    import pickle 

    BUCKET_NAME = 'xxxx'
    MODEL_FILE_NAME = 'text-classification-model-v1.pk' 
    VECTORIZER_FILE_NAME = 'word_vectorizer.pk'
    ACCESS_KEY="QQQQ"
    SECRET_KEY="ZZZZZ"
    S3 = boto3.client('s3', aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY, region_name='us-west-2')
    app = Flask(__name__)

    @app.route('/', methods=['GET'])
    def index():
        payload = [request.args.get('description')]     
        prediction = predict_(payload)
        return parse_data()

    def predict_(payload):
        transformed_text_features = load_vectorizer(payload)
        return load_model(MODEL_FILE_NAME).predict(transformed_text_features) 

    def parse_data():
        return 'xxx'

    def load_vectorizer(payload):
        with open("downloaded_vectorizer.pk", 'wb') as data:
            S3.download_fileobj(BUCKET_NAME, VECTORIZER_FILE_NAME, data)
        infile = open("downloaded_vectorizer.pk","rb")
        loaded_vectorizer = joblib.load(infile)  
        infile.close()
        transformed_text_features = loaded_vectorizer.transform(payload)  
        return transformed_text_features

    def load_model(model):
        with open("downloaded_model_textclassification.pkl", 'wb') as data:  
            S3.download_fileobj(BUCKET_NAME, MODEL_FILE_NAME, data)
        infile = open("downloaded_model_textclassification.pkl","rb")    
        model = pickle.load(infile) 
        infile.close()
        return model 

    ==========
    error message
    ==========
      File "\flask\predict_commodity.py", line 32, in index
        prediction = predict_(payload)
      File "\flask\predict_commodity.py", line 38, in predict_
        transformed_text_features = load_vectorizer(payload)
      File "\flask\predict_commodity.py", line 60, in load_vectorize
        loaded_vectorizer = joblib.load(infile)
      File " \lib\site-packages\sklearn\externals\joblib\numpy_pickle.py", line 568, in load
        obj = _unpickle(fobj)
      File " \lib\site-packages\sklearn\externals\joblib\numpy_pickle.py", line 508, in _unpickle
        obj = unpickler.load()
      File " \anaconda3\Lib\pickle.py", line 1050, in load
        dispatch[key[0]](self)
      File "\Lib\pickle.py", line 1338, in load_global
        klass = self.find_class(module, name)
      File "\Lib\pickle.py", line 1388, in find_class
        __import__(module, level=0)
    ModuleNotFoundError: No module named 'dill.dill'

Answer 1

在您的代码中，您使用的是 pickle 和 sklearn.externals.joblib。这可能是个问题。

另外，您可以尝试更新包：pip install dill --upgrade。这帮助我解决了同样的问题。

加载保存的文件时 Flask 出错并显示 dill.dill 消息

Flask error out with dill.dill message when loading saved file

python

machine-learning

amazon-s3

pickle

dill