Cython "ModuleNotFoundError" 即使它是在构建 docker 图像时通过 requirements.txt 安装的
Cython "ModuleNotFoundError" even though it was just installed via requirements.txt while building docker image
在通过 requirements.txt 构建 docker 图像时安装包。在 Dockerfile 中它显示为
COPY requirements.txt /app/requirements.txt
RUN pip3 install -r /app/requirements.txt
在 requirements.txt 中，我放下了用于安装私有 git 存储库的部署令牌，例如这个示例一：
git+https://gitlab+deploy-token-123456:a1b2c3d4e5f6g7h8j9@gitlab.com/example/lib.git@a1b2c3d4e5f6g7h8j9k1l2m3n4o5p6q7r8s9t
它抛出一个错误，ModuleNotFoundError: No module named 'Cython'。
然而，即使我在要求中包含了 Cython 并且它被正确收集，我也得到了同样的错误。
完整消息如下：
docker build -t docker.example/example:dev .
Sending build context to Docker daemon 8.704kB
Step 1/7 : FROM python:3.6.6-alpine
---> a78e257617d1
Step 2/7 : RUN apk update
---> Using cache
---> 35e6b7308e61
Step 3/7 : RUN apk add gcc build-base musl-dev libc-dev util-linux-dev linux-headers python3-dev git ca-certificates libffi-dev bash postgresql-libs postgresql-dev
---> Using cache
---> 5ca54839bb7c
Step 4/7 : RUN python3 -m ensurepip
---> Using cache
---> 03d95f700108
Step 5/7 : RUN pip install --upgrade pip
---> Using cache
---> e462d855f07f
Step 6/7 : COPY requirements.txt /app/requirements.txt
---> f69aa355732a
Step 7/7 : RUN pip3 install -r /app/requirements.txt
---> Running in d75911af46c7
Collecting git+https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git@f75285be6ee56360d6f0cfbf64f09a86819def5c (from -r /app/requirements.txt (line 2))
Cloning https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git (to revision f75285be6ee56360d6f0cfbf64f09a86819def5c) to /tmp/pip-req-build-yt5mk7q5
Running command git clone -q 'https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git' /tmp/pip-req-build-yt5mk7q5
Collecting Cython==0.29.17
Downloading Cython-0.29.17-py2.py3-none-any.whl (971 kB)
Collecting psychrolib==2.5.0
Downloading PsychroLib-2.5.0.zip (9.5 kB)
Collecting influxdb==5.2.2
Downloading influxdb-5.2.2-py2.py3-none-any.whl (70 kB)
Collecting configparser==3.5.0
Downloading configparser-3.5.0.tar.gz (39 kB)
Collecting reportlab
Downloading reportlab-3.5.42.tar.gz (2.9 MB)
Collecting PyPDF2
Downloading PyPDF2-1.26.0.tar.gz (77 kB)
Collecting matplotlib>=2.2.2
Downloading matplotlib-3.2.1.tar.gz (40.3 MB)
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-mqz7vwwc/matplotlib/pip-egg-info
cwd: /tmp/pip-install-mqz7vwwc/matplotlib/
Complete output (112 lines):
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_bounded_integers.pyx.in
Traceback (most recent call last):
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 61, in process_pyx
from Cython.Compiler.Version import version as cython_version
ModuleNotFoundError: No module named 'Cython'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 238, in <module>
main()
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 234, in main
find_process_files(root_dir)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 225, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 191, in process
processor_function(fromfile, tofile)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 94, in process_tempita_pyx
process_pyx(pyxfile, tofile)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 66, in process_pyx
raise OSError('Cython needs to be installed in Python as a module')
OSError: Cython needs to be installed in Python as a module
Running from numpy source directory.
/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
run_build = parse_setuppy_commands()
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
)
RuntimeError: Running cythonize failed!
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-mqz7vwwc/matplotlib/setup.py", line 274, in <module>
cmdclass=cmdclass,
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup
_install_setup_requires(attrs)
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve
replace_conflicting=replace_conflicting
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain
return installer(requirement)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 676, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 702, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 887, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1155, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1141, in run_setup
run_setup(setup_script, args)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 253, in run_setup
raise
File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/local/lib/python3.6/site-packages/setuptools/_vendor/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
)
RuntimeError: Running cythonize failed!
Edit setup.cfg to change the build options; suppress output with --quiet.
BUILDING MATPLOTLIB
matplotlib: yes [3.2.1]
python: yes [3.6.6 (default, Sep 12 2018, 02:15:29) [GCC 6.4.0]]
platform: yes [linux]
sample_data: yes [installing]
tests: no [skipping due to configuration]
agg: yes [installing]
tkagg: yes [installing; run-time loading from Python Tcl/Tk]
macosx: no [Mac OS-X only]
Cythonizing sources
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.0.2; however, version 20.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
The command '/bin/sh -c pip3 install -r /app/requirements.txt' returned a non-zero code: 1
Makefile:8: recipe for target 'build' failed
make: *** [build] Error 1
所以问题是你安装了两个版本的 Python，我猜：
- Docker 图片包含 Python。
- 然后您安装了 Alpine 系统 Python
apk。
您应该从 apk 安装行删除 python-dev 包。
此外——你只需要编译 matplotlib、numpy 等，因为你使用的是 Alpine。如果您切换到不是基于 Alpine 的基础映像（例如 python:3.6-slim-buster），您将能够使用预编译的轮子，并且您的构建过程会更快。请参阅 https://pythonspeed.com/articles/alpine-docker-python/ 进行更长时间的讨论，但基本上 Alpine 使用的 libc 与标准 Linux glibc 不同，因此 PyPI 的轮子不起作用。
我能够通过包含
来解决问题
RUN pip3 install Cython
在Dockerfile中，上面的要求安装。
在通过 requirements.txt 构建 docker 图像时安装包。在 Dockerfile 中它显示为
COPY requirements.txt /app/requirements.txt
RUN pip3 install -r /app/requirements.txt
在 requirements.txt 中，我放下了用于安装私有 git 存储库的部署令牌，例如这个示例一：
git+https://gitlab+deploy-token-123456:a1b2c3d4e5f6g7h8j9@gitlab.com/example/lib.git@a1b2c3d4e5f6g7h8j9k1l2m3n4o5p6q7r8s9t
它抛出一个错误，ModuleNotFoundError: No module named 'Cython'。
然而，即使我在要求中包含了 Cython 并且它被正确收集，我也得到了同样的错误。
完整消息如下：
docker build -t docker.example/example:dev .
Sending build context to Docker daemon 8.704kB
Step 1/7 : FROM python:3.6.6-alpine
---> a78e257617d1
Step 2/7 : RUN apk update
---> Using cache
---> 35e6b7308e61
Step 3/7 : RUN apk add gcc build-base musl-dev libc-dev util-linux-dev linux-headers python3-dev git ca-certificates libffi-dev bash postgresql-libs postgresql-dev
---> Using cache
---> 5ca54839bb7c
Step 4/7 : RUN python3 -m ensurepip
---> Using cache
---> 03d95f700108
Step 5/7 : RUN pip install --upgrade pip
---> Using cache
---> e462d855f07f
Step 6/7 : COPY requirements.txt /app/requirements.txt
---> f69aa355732a
Step 7/7 : RUN pip3 install -r /app/requirements.txt
---> Running in d75911af46c7
Collecting git+https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git@f75285be6ee56360d6f0cfbf64f09a86819def5c (from -r /app/requirements.txt (line 2))
Cloning https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git (to revision f75285be6ee56360d6f0cfbf64f09a86819def5c) to /tmp/pip-req-build-yt5mk7q5
Running command git clone -q 'https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git' /tmp/pip-req-build-yt5mk7q5
Collecting Cython==0.29.17
Downloading Cython-0.29.17-py2.py3-none-any.whl (971 kB)
Collecting psychrolib==2.5.0
Downloading PsychroLib-2.5.0.zip (9.5 kB)
Collecting influxdb==5.2.2
Downloading influxdb-5.2.2-py2.py3-none-any.whl (70 kB)
Collecting configparser==3.5.0
Downloading configparser-3.5.0.tar.gz (39 kB)
Collecting reportlab
Downloading reportlab-3.5.42.tar.gz (2.9 MB)
Collecting PyPDF2
Downloading PyPDF2-1.26.0.tar.gz (77 kB)
Collecting matplotlib>=2.2.2
Downloading matplotlib-3.2.1.tar.gz (40.3 MB)
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-mqz7vwwc/matplotlib/pip-egg-info
cwd: /tmp/pip-install-mqz7vwwc/matplotlib/
Complete output (112 lines):
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_bounded_integers.pyx.in
Traceback (most recent call last):
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 61, in process_pyx
from Cython.Compiler.Version import version as cython_version
ModuleNotFoundError: No module named 'Cython'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 238, in <module>
main()
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 234, in main
find_process_files(root_dir)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 225, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 191, in process
processor_function(fromfile, tofile)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 94, in process_tempita_pyx
process_pyx(pyxfile, tofile)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 66, in process_pyx
raise OSError('Cython needs to be installed in Python as a module')
OSError: Cython needs to be installed in Python as a module
Running from numpy source directory.
/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
run_build = parse_setuppy_commands()
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
)
RuntimeError: Running cythonize failed!
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-mqz7vwwc/matplotlib/setup.py", line 274, in <module>
cmdclass=cmdclass,
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup
_install_setup_requires(attrs)
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve
replace_conflicting=replace_conflicting
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain
return installer(requirement)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 676, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 702, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 887, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1155, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1141, in run_setup
run_setup(setup_script, args)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 253, in run_setup
raise
File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/local/lib/python3.6/site-packages/setuptools/_vendor/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
)
RuntimeError: Running cythonize failed!
Edit setup.cfg to change the build options; suppress output with --quiet.
BUILDING MATPLOTLIB
matplotlib: yes [3.2.1]
python: yes [3.6.6 (default, Sep 12 2018, 02:15:29) [GCC 6.4.0]]
platform: yes [linux]
sample_data: yes [installing]
tests: no [skipping due to configuration]
agg: yes [installing]
tkagg: yes [installing; run-time loading from Python Tcl/Tk]
macosx: no [Mac OS-X only]
Cythonizing sources
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.0.2; however, version 20.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
The command '/bin/sh -c pip3 install -r /app/requirements.txt' returned a non-zero code: 1
Makefile:8: recipe for target 'build' failed
make: *** [build] Error 1
所以问题是你安装了两个版本的 Python，我猜：
- Docker 图片包含 Python。
- 然后您安装了 Alpine 系统 Python
apk。
您应该从 apk 安装行删除 python-dev 包。
此外——你只需要编译 matplotlib、numpy 等，因为你使用的是 Alpine。如果您切换到不是基于 Alpine 的基础映像（例如 python:3.6-slim-buster），您将能够使用预编译的轮子，并且您的构建过程会更快。请参阅 https://pythonspeed.com/articles/alpine-docker-python/ 进行更长时间的讨论，但基本上 Alpine 使用的 libc 与标准 Linux glibc 不同，因此 PyPI 的轮子不起作用。
我能够通过包含
来解决问题RUN pip3 install Cython
在Dockerfile中，上面的要求安装。