加载时找不到 Pickle 文件

Question

我使用以下代码保存了一个带有 pickle 的分类器模型-

import pickle
with open('skm.pickle', 'wb') as fid:
    pickle.dump(clf, fid)

这是 score.py 文件，我在其中加载 pickle 模型，并且在部署期间调用了同一文件。另请注意，所有这些文件（代码、pickle 文件和相关文件）都在同一目录中。

%%writefile sklearnscore.py

import json
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.ensemble import RandomForestClassifier
import pickle

# Initialize the deployment environment
def init():
    # read in the model file
    from sklearn.pipeline import Pipeline
    global obj
    with open('./skm.pickle', 'rb') as f:
        obj = pickle.load(f)

这正在存储在某个目录中 var/azureml-app

我正在使用 - model = Model.register(ws, model_name="utility15", model_path="./skm.pickle")

注册模型

而部署代码是-

service_name = 'my-custom-env-service-4'
sklearn_env = Environment.from_conda_specification(name='sklearn-env', file_path='Sklearn.yaml')

inference_config = InferenceConfig(entry_script='sklearnscore.py', environment=sklearn_env)

aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=4,tags={'Createdby':'xyz'})

service = Model.deploy(workspace=ws,
                        name=service_name,
                        models=[model],
                        inference_config=inference_config,
                        deployment_config=aci_config,
                        overwrite=True)
service.wait_for_deployment(show_output=True)

在运行这个部署代码之后，这是我得到的错误-

2022-05-24 09:58:08,198 | root | INFO | Starting up app insights client
logging socket was found. logging is available.
logging socket was found. logging is available.
2022-05-24 09:58:08,199 | root | INFO | Starting up request id generator
2022-05-24 09:58:08,199 | root | INFO | Starting up app insight hooks
2022-05-24 09:58:08,199 | root | INFO | Invoking user's init function
2022-05-24 09:58:08,201 | root | ERROR | User's init function failed
2022-05-24 09:58:08,203 | root | ERROR | Encountered Exception Traceback (most recent call last):
  File "/var/azureml-server/aml_blueprint.py", line 191, in register
    main.init()
  File "/var/azureml-app/sklearnscore.py", line 11, in init
    with open('/mnt/batch/tasks/shared/LS_root/mounts/clusters/lowcomputeamber/code/Users/1964821/skm.pickle', 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/batch/tasks/shared/LS_root/mounts/clusters/lowcomputeamber/code/Users/1964821/skm.pickle'

2022-05-24 09:58:08,204 | root | INFO | Waiting for logs to be sent to Application Insights before exit.
2022-05-24 09:58:08,205 | root | INFO | Waiting 30 seconds for upload.
Worker exiting (pid: 103)
Shutting down: Master
Reason: Worker failed to boot.
2022-05-24T09:58:38,514243200+00:00 - gunicorn/finish 3 0
2022-05-24T09:58:38,516894700+00:00 - Exit code 3 is not normal. Killing image.

从这个错误中，我可以看到 sklearnscore.py 在 /var/azureml-app/sklearnscore.py 处，因此，它可能期望 pickle 模型在那里。但是，由于路径不可访问，我该如何将泡菜模型保存在路径 var/azureml-app 上？

Answer 1

我想出了解决这个错误的方法。

而不是将模型加载为

 with open('./skm.pickle', 'rb') as f:
        obj = pickle.load(f)

您需要将模型加载为

 model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'skm.pickle')
 model = joblib.load(model_path)

AZUREML_MODEL_DIR 将采用存储模型的环境目录。

这将解决错误。

加载时找不到 Pickle 文件

Pickle File not found while loading

python

pickle

azure

python-3.x

scikit-learn