ModuleNotFoundError: No module named 'numpy.random._pickle'

Question

我有一个 doc2vec 模型来驱动我的推荐应用程序。我已经构建了 doc2vec 模型并保存到 s3 存储桶中。现在，当我打开 webapp 时，模型应该从 s3 加载回来，但这并没有发生。我使用 AWS Elasticbean stalk 来部署我的网络应用程序。因此，我随后检查了 Elasticbean stalk 的日志是否有错误。这是我尝试部署时显示的错误

-------------------------------------
/var/log/httpd/error_log
-------------------------------------
[Fri May 22 12:53:57.768095 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:53:57.768098 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     klass = self.find_class(module, name)
[Fri May 22 12:53:57.768104 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:53:57.768107 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     __import__(module, level=0)
[Fri May 22 12:53:57.768127 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:53:57.768138 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] 
[Fri May 22 12:53:57.768141 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] During handling of the above exception, another exception occurred:
[Fri May 22 12:53:57.768144 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] 
[Fri May 22 12:53:57.768150 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] Traceback (most recent call last):
[Fri May 22 12:53:57.768177 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/opt/python/current/app/application.py", line 81, in <module>
[Fri May 22 12:53:57.768181 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     model_d2v = d2v.load_d2v('doc2vec_model', ENV)
[Fri May 22 12:53:57.768187 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/opt/python/current/app/microservices/d2v_services.py", line 21, in load_d2v
[Fri May 22 12:53:57.768190 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     model=joblib.load(model_name)
[Fri May 22 12:53:57.768196 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 605, in load
[Fri May 22 12:53:57.768199 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     obj = _unpickle(fobj, filename, mmap_mode)
[Fri May 22 12:53:57.768204 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 529, in _unpickle
[Fri May 22 12:53:57.768208 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     obj = unpickler.load()
[Fri May 22 12:53:57.768213 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/usr/lib64/python3.6/pickle.py", line 1050, in load
[Fri May 22 12:53:57.768216 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     dispatch[key[0]](self)
[Fri May 22 12:53:57.768222 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:53:57.768230 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     klass = self.find_class(module, name)
[Fri May 22 12:53:57.768237 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]   File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:53:57.768240 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]     __import__(module, level=0)
[Fri May 22 12:53:57.768250 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:54:07.192783 2020] [:error] [pid 8177] loading...doc2vec_model
[Fri May 22 12:54:12.237091 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] mod_wsgi (pid=8177): Target WSGI script '/opt/python/current/app/application.py' cannot be loaded as Python module.
[Fri May 22 12:54:12.237334 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] mod_wsgi (pid=8177): Exception occurred processing WSGI script '/opt/python/current/app/application.py'.
[Fri May 22 12:54:12.238604 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] Traceback (most recent call last):
[Fri May 22 12:54:12.238658 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/opt/python/current/app/microservices/d2v_services.py", line 14, in load_d2v
[Fri May 22 12:54:12.238664 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     model=joblib.load(model_name)
[Fri May 22 12:54:12.238675 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 605, in load
[Fri May 22 12:54:12.238679 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     obj = _unpickle(fobj, filename, mmap_mode)
[Fri May 22 12:54:12.238686 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 529, in _unpickle
[Fri May 22 12:54:12.238690 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     obj = unpickler.load()
[Fri May 22 12:54:12.238695 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/usr/lib64/python3.6/pickle.py", line 1050, in load
[Fri May 22 12:54:12.238699 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     dispatch[key[0]](self)
[Fri May 22 12:54:12.238705 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:54:12.238709 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     klass = self.find_class(module, name)
[Fri May 22 12:54:12.238714 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:54:12.238718 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     __import__(module, level=0)
[Fri May 22 12:54:12.238737 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:54:12.238793 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] 
[Fri May 22 12:54:12.238800 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] During handling of the above exception, another exception occurred:
[Fri May 22 12:54:12.238803 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] 
[Fri May 22 12:54:12.238810 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] Traceback (most recent call last):
[Fri May 22 12:54:12.238840 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/opt/python/current/app/application.py", line 81, in <module>
[Fri May 22 12:54:12.238845 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     model_d2v = d2v.load_d2v('doc2vec_model', ENV)
[Fri May 22 12:54:12.238851 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/opt/python/current/app/microservices/d2v_services.py", line 21, in load_d2v
[Fri May 22 12:54:12.238855 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     model=joblib.load(model_name)
[Fri May 22 12:54:12.238872 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 605, in load
[Fri May 22 12:54:12.238883 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     obj = _unpickle(fobj, filename, mmap_mode)
[Fri May 22 12:54:12.238889 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 529, in _unpickle
[Fri May 22 12:54:12.238892 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     obj = unpickler.load()
[Fri May 22 12:54:12.238897 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/usr/lib64/python3.6/pickle.py", line 1050, in load
[Fri May 22 12:54:12.238901 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     dispatch[key[0]](self)
[Fri May 22 12:54:12.238906 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:54:12.238909 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     klass = self.find_class(module, name)
[Fri May 22 12:54:12.238914 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]   File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:54:12.238917 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]     __import__(module, level=0)
[Fri May 22 12:54:12.238928 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:56:22.918445 2020] [mpm_prefork:notice] [pid 8172] AH00169: caught SIGTERM, shutting down

我尝试导入 'numpy.random._pickle' 但没有成功。

我用来加载模型的代码：

import subprocess
from sklearn.externals import joblib
import gensim

model = load_d2v("doc2vec_model")

def load_d2v(fname):
    model_name = fname
    s3_base_path='s3://sd-flikku/datalake/current_doc2vec_model'
    path = s3_base_path+'/'+model_name  
    command = "aws s3 cp {} {}".format(path,model_name).split()
    print('loading...'+model_name)
    subprocess.call(command)
    model=joblib.load(model_name)
    return model

你能帮我解决这个问题吗？提前致谢。

Answer 1

这很可能是 numpy 版本不匹配。

这可能来自未完全卸载的旧版本或您正在使用的软件包版本错误。

These are people with the same problem

Answer 2

使用以下方式升级 numpy 库：

pip install numpy --upgrade

当使用特定版本的库构建模型并使用其他版本测试时发生。

ModuleNotFoundError: No module named 'numpy.random._pickle'

ModuleNotFoundError: No module named 'numpy.random._pickle'

numpy

pickle

python-3.x

joblib

doc2vec