docker 构建期间没有名为 'numpy' 的模块
No module named 'numpy' during docker build
我按照说明 (https://github.com/huggingface/transfer-learning-conv-ai) 从 huggingface 安装 conv-ai，但我卡在 docker 构建步骤：docker build -t convai .
我用的是Mac10.15，python3.8，增加了Docker内存到4G。
我尝试了以下方法来解决问题：
- 在
requirements.txt 中添加numpy
- 在Docker文件中添加
RUN pip3 install --upgrade setuptools
- 将
--upgrade 添加到 Docker 文件中的 RUN pip3 install -r /tmp/requirements.txt
- 在Docker文件
中的RUN pip3 install -r /tmp/requirements.txt之前添加RUN pip3 install numpy
- 在Docker文件
中的RUN pip3 install -r /tmp/requirements.txt之前添加RUN apt-get install python3-numpy
- 使用 python 3.6.13 因为这个 post，但它有完全相同的错误。
- 我目前正在通过在
RUN pip3 install requirements.txt 之前输入来调试容器内部
谁能帮我解决这个问题？谢谢！！
错误：
=> [6/9] COPY . ./ 0.0s
=> [7/9] COPY requirements.txt /tmp/requirements.txt 0.0s
=> ERROR [8/9] RUN pip3 install -r /tmp/requirements.txt 98.2s
------
> [8/9] RUN pip3 install -r /tmp/requirements.txt:
#12 1.111 Collecting torch (from -r /tmp/requirements.txt (line 1))
#12 1.754 Downloading https://files.pythonhosted.org/packages/46/99/8b658e5095b9fb02e38ccb7ecc931eb1a03b5160d77148aecf68f8a7eeda/torch-1.8.0-cp36-cp36m-manylinux1_x86_64.whl (735.5MB)
#12 81.11 Collecting pytorch-ignite (from -r /tmp/requirements.txt (line 2))
#12 81.76 Downloading https://files.pythonhosted.org/packages/f8/d3/640f70d69393b415e6a29b27c735047ad86267921ad62682d1d756556d48/pytorch_ignite-0.4.4-py3-none-any.whl (200kB)
#12 81.82 Collecting transformers==2.5.1 (from -r /tmp/requirements.txt (line 3))
#12 82.17 Downloading https://files.pythonhosted.org/packages/13/33/ffb67897a6985a7b7d8e5e7878c3628678f553634bd3836404fef06ef19b/transformers-2.5.1-py3-none-any.whl (499kB)
#12 82.29 Collecting tensorboardX==1.8 (from -r /tmp/requirements.txt (line 4))
#12 82.50 Downloading https://files.pythonhosted.org/packages/c3/12/dcaf67e1312475b26db9e45e7bb6f32b540671a9ee120b3a72d9e09bc517/tensorboardX-1.8-py2.py3-none-any.whl (216kB)
#12 82.57 Collecting tensorflow (from -r /tmp/requirements.txt (line 5))
#12 83.12 Downloading https://files.pythonhosted.org/packages/de/f0/96fb2e0412ae9692dbf400e5b04432885f677ad6241c088ccc5fe7724d69/tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2MB)
#12 95.24 Collecting spacy (from -r /tmp/requirements.txt (line 6))
#12 95.81 Downloading https://files.pythonhosted.org/packages/65/01/fd65769520d4b146d92920170fd00e01e826cda39a366bde82a87ca249db/spacy-3.0.5.tar.gz (7.0MB)
#12 97.41 Complete output from command python setup.py egg_info:
#12 97.41 Traceback (most recent call last):
#12 97.41 File "<string>", line 1, in <module>
#12 97.41 File "/tmp/pip-build-cc3a804w/spacy/setup.py", line 5, in <module>
#12 97.41 import numpy
#12 97.41 ModuleNotFoundError: No module named 'numpy'
#12 97.41
#12 97.41 ----------------------------------------
#12 98.11 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-cc3a804w/spacy/
@Håken Lid 如果我 RUN pip3 install numpy 就在 RUN pip3 install -r tmp/requirements 之前得到的错误：
=> [ 8/10] RUN pip3 install numpy 10.1s
=> ERROR [ 9/10] RUN pip3 install -r /tmp/requirements.txt 112.4s
------
> [ 9/10] RUN pip3 install -r /tmp/requirements.txt:
#13 1.067 Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from -r /tmp/requirements.txt (line 1))
#13 1.074 Collecting torch (from -r /tmp/requirements.txt (line 2))
#13 1.656 Downloading https://files.pythonhosted.org/packages/46/99/8b658e5095b9fb02e38ccb7ecc931eb1a03b5160d77148aecf68f8a7eeda/torch-1.8.0-cp36-cp36m-manylinux1_x86_64.whl (735.5MB)
#13 96.46 Collecting pytorch-ignite (from -r /tmp/requirements.txt (line 3))
#13 97.02 Downloading https://files.pythonhosted.org/packages/f8/d3/640f70d69393b415e6a29b27c735047ad86267921ad62682d1d756556d48/pytorch_ignite-0.4.4-py3-none-any.whl (200kB)
#13 97.07 Collecting transformers==2.5.1 (from -r /tmp/requirements.txt (line 4))
#13 97.32 Downloading https://files.pythonhosted.org/packages/13/33/ffb67897a6985a7b7d8e5e7878c3628678f553634bd3836404fef06ef19b/transformers-2.5.1-py3-none-any.whl (499kB)
#13 97.43 Collecting tensorboardX==1.8 (from -r /tmp/requirements.txt (line 5))
#13 97.70 Downloading https://files.pythonhosted.org/packages/c3/12/dcaf67e1312475b26db9e45e7bb6f32b540671a9ee120b3a72d9e09bc517/tensorboardX-1.8-py2.py3-none-any.whl (216kB)
#13 97.76 Collecting tensorflow (from -r /tmp/requirements.txt (line 6))
#13 98.27 Downloading https://files.pythonhosted.org/packages/de/f0/96fb2e0412ae9692dbf400e5b04432885f677ad6241c088ccc5fe7724d69/tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2MB)
#13 109.6 Collecting spacy (from -r /tmp/requirements.txt (line 7))
#13 110.0 Downloading https://files.pythonhosted.org/packages/65/01/fd65769520d4b146d92920170fd00e01e826cda39a366bde82a87ca249db/spacy-3.0.5.tar.gz (7.0MB)
#13 111.6 Complete output from command python setup.py egg_info:
#13 111.6 Traceback (most recent call last):
#13 111.6 File "<string>", line 1, in <module>
#13 111.6 File "/tmp/pip-build-t6n57csv/spacy/setup.py", line 10, in <module>
#13 111.6 from Cython.Build import cythonize
#13 111.6 ModuleNotFoundError: No module named 'Cython'
#13 111.6
#13 111.6 ----------------------------------------
#13 112.3 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-t6n57csv/spacy/
------
executor failed running [/bin/sh -c pip3 install -r /tmp/requirements.txt]: exit code: 1
requirements.txt:
torch
pytorch-ignite
transformers==2.5.1
tensorboardX==1.8
tensorflow # for tensorboardX
spacy
Docker文件:
FROM ubuntu:18.04
MAINTAINER Loreto Parisi loretoparisi@gmail.com
######################################## BASE SYSTEM
# set noninteractive installation
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y apt-utils
RUN apt-get install -y --no-install-recommends \
build-essential \
pkg-config \
tzdata \
curl
######################################## PYTHON3
RUN apt-get install -y \
python3 \
python3-pip
# set local timezone
RUN ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime && \
dpkg-reconfigure --frontend noninteractive tzdata
# transfer-learning-conv-ai
ENV PYTHONPATH /usr/local/lib/python3.6
COPY . ./
COPY requirements.txt /tmp/requirements.txt
RUN pip3 install -r /tmp/requirements.txt
# model zoo
RUN mkdir models && \
curl https://s3.amazonaws.com/models.huggingface.co/transfer-learning-chatbot/finetuned_chatbot_gpt.tar.gz > models/finetuned_chatbot_gpt.tar.gz && \
cd models/ && \
tar -xvzf finetuned_chatbot_gpt.tar.gz && \
rm finetuned_chatbot_gpt.tar.gz
CMD ["bash"]
我 运行 到目前为止的步骤：
git clone https://github.com/huggingface/transfer-learning-conv-ai
cd transfer-learning-conv-ai
pip install -r requirements.txt
python -m spacy download en
docker build -t convai .
您是否尝试将 numpy 添加到 requirements.txt？在我看来它不见了。
pip 似乎没有安装预制轮子，而是尝试从源代码构建 spacy。这是一个脆弱的过程，需要额外的依赖。
为避免这种情况，您应该确保 Python 软件包 pip、wheel 和 setuptools 在继续安装之前是最新的。
# replace RUN pip3 install -r /tmp/requirements.txt
RUN python3 -m pip install --upgrade pip setuptools wheel
RUN python3 -m pip install -r /tmp/requirements.txt
我按照说明 (https://github.com/huggingface/transfer-learning-conv-ai) 从 huggingface 安装 conv-ai，但我卡在 docker 构建步骤：docker build -t convai .
我用的是Mac10.15，python3.8，增加了Docker内存到4G。
我尝试了以下方法来解决问题：
- 在
requirements.txt中添加 - 在Docker文件中添加
RUN pip3 install --upgrade setuptools - 将
--upgrade添加到 Docker 文件中的RUN pip3 install -r /tmp/requirements.txt - 在Docker文件 中的
- 在Docker文件 中的
- 使用 python 3.6.13 因为这个 post，但它有完全相同的错误。
- 我目前正在通过在
RUN pip3 install requirements.txt 之前输入来调试容器内部
numpy
RUN pip3 install -r /tmp/requirements.txt之前添加RUN pip3 install numpy
RUN pip3 install -r /tmp/requirements.txt之前添加RUN apt-get install python3-numpy
谁能帮我解决这个问题？谢谢！！
错误：
=> [6/9] COPY . ./ 0.0s
=> [7/9] COPY requirements.txt /tmp/requirements.txt 0.0s
=> ERROR [8/9] RUN pip3 install -r /tmp/requirements.txt 98.2s
------
> [8/9] RUN pip3 install -r /tmp/requirements.txt:
#12 1.111 Collecting torch (from -r /tmp/requirements.txt (line 1))
#12 1.754 Downloading https://files.pythonhosted.org/packages/46/99/8b658e5095b9fb02e38ccb7ecc931eb1a03b5160d77148aecf68f8a7eeda/torch-1.8.0-cp36-cp36m-manylinux1_x86_64.whl (735.5MB)
#12 81.11 Collecting pytorch-ignite (from -r /tmp/requirements.txt (line 2))
#12 81.76 Downloading https://files.pythonhosted.org/packages/f8/d3/640f70d69393b415e6a29b27c735047ad86267921ad62682d1d756556d48/pytorch_ignite-0.4.4-py3-none-any.whl (200kB)
#12 81.82 Collecting transformers==2.5.1 (from -r /tmp/requirements.txt (line 3))
#12 82.17 Downloading https://files.pythonhosted.org/packages/13/33/ffb67897a6985a7b7d8e5e7878c3628678f553634bd3836404fef06ef19b/transformers-2.5.1-py3-none-any.whl (499kB)
#12 82.29 Collecting tensorboardX==1.8 (from -r /tmp/requirements.txt (line 4))
#12 82.50 Downloading https://files.pythonhosted.org/packages/c3/12/dcaf67e1312475b26db9e45e7bb6f32b540671a9ee120b3a72d9e09bc517/tensorboardX-1.8-py2.py3-none-any.whl (216kB)
#12 82.57 Collecting tensorflow (from -r /tmp/requirements.txt (line 5))
#12 83.12 Downloading https://files.pythonhosted.org/packages/de/f0/96fb2e0412ae9692dbf400e5b04432885f677ad6241c088ccc5fe7724d69/tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2MB)
#12 95.24 Collecting spacy (from -r /tmp/requirements.txt (line 6))
#12 95.81 Downloading https://files.pythonhosted.org/packages/65/01/fd65769520d4b146d92920170fd00e01e826cda39a366bde82a87ca249db/spacy-3.0.5.tar.gz (7.0MB)
#12 97.41 Complete output from command python setup.py egg_info:
#12 97.41 Traceback (most recent call last):
#12 97.41 File "<string>", line 1, in <module>
#12 97.41 File "/tmp/pip-build-cc3a804w/spacy/setup.py", line 5, in <module>
#12 97.41 import numpy
#12 97.41 ModuleNotFoundError: No module named 'numpy'
#12 97.41
#12 97.41 ----------------------------------------
#12 98.11 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-cc3a804w/spacy/
@Håken Lid 如果我 RUN pip3 install numpy 就在 RUN pip3 install -r tmp/requirements 之前得到的错误：
=> [ 8/10] RUN pip3 install numpy 10.1s
=> ERROR [ 9/10] RUN pip3 install -r /tmp/requirements.txt 112.4s
------
> [ 9/10] RUN pip3 install -r /tmp/requirements.txt:
#13 1.067 Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from -r /tmp/requirements.txt (line 1))
#13 1.074 Collecting torch (from -r /tmp/requirements.txt (line 2))
#13 1.656 Downloading https://files.pythonhosted.org/packages/46/99/8b658e5095b9fb02e38ccb7ecc931eb1a03b5160d77148aecf68f8a7eeda/torch-1.8.0-cp36-cp36m-manylinux1_x86_64.whl (735.5MB)
#13 96.46 Collecting pytorch-ignite (from -r /tmp/requirements.txt (line 3))
#13 97.02 Downloading https://files.pythonhosted.org/packages/f8/d3/640f70d69393b415e6a29b27c735047ad86267921ad62682d1d756556d48/pytorch_ignite-0.4.4-py3-none-any.whl (200kB)
#13 97.07 Collecting transformers==2.5.1 (from -r /tmp/requirements.txt (line 4))
#13 97.32 Downloading https://files.pythonhosted.org/packages/13/33/ffb67897a6985a7b7d8e5e7878c3628678f553634bd3836404fef06ef19b/transformers-2.5.1-py3-none-any.whl (499kB)
#13 97.43 Collecting tensorboardX==1.8 (from -r /tmp/requirements.txt (line 5))
#13 97.70 Downloading https://files.pythonhosted.org/packages/c3/12/dcaf67e1312475b26db9e45e7bb6f32b540671a9ee120b3a72d9e09bc517/tensorboardX-1.8-py2.py3-none-any.whl (216kB)
#13 97.76 Collecting tensorflow (from -r /tmp/requirements.txt (line 6))
#13 98.27 Downloading https://files.pythonhosted.org/packages/de/f0/96fb2e0412ae9692dbf400e5b04432885f677ad6241c088ccc5fe7724d69/tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2MB)
#13 109.6 Collecting spacy (from -r /tmp/requirements.txt (line 7))
#13 110.0 Downloading https://files.pythonhosted.org/packages/65/01/fd65769520d4b146d92920170fd00e01e826cda39a366bde82a87ca249db/spacy-3.0.5.tar.gz (7.0MB)
#13 111.6 Complete output from command python setup.py egg_info:
#13 111.6 Traceback (most recent call last):
#13 111.6 File "<string>", line 1, in <module>
#13 111.6 File "/tmp/pip-build-t6n57csv/spacy/setup.py", line 10, in <module>
#13 111.6 from Cython.Build import cythonize
#13 111.6 ModuleNotFoundError: No module named 'Cython'
#13 111.6
#13 111.6 ----------------------------------------
#13 112.3 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-t6n57csv/spacy/
------
executor failed running [/bin/sh -c pip3 install -r /tmp/requirements.txt]: exit code: 1
requirements.txt:
torch
pytorch-ignite
transformers==2.5.1
tensorboardX==1.8
tensorflow # for tensorboardX
spacy
Docker文件:
FROM ubuntu:18.04
MAINTAINER Loreto Parisi loretoparisi@gmail.com
######################################## BASE SYSTEM
# set noninteractive installation
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y apt-utils
RUN apt-get install -y --no-install-recommends \
build-essential \
pkg-config \
tzdata \
curl
######################################## PYTHON3
RUN apt-get install -y \
python3 \
python3-pip
# set local timezone
RUN ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime && \
dpkg-reconfigure --frontend noninteractive tzdata
# transfer-learning-conv-ai
ENV PYTHONPATH /usr/local/lib/python3.6
COPY . ./
COPY requirements.txt /tmp/requirements.txt
RUN pip3 install -r /tmp/requirements.txt
# model zoo
RUN mkdir models && \
curl https://s3.amazonaws.com/models.huggingface.co/transfer-learning-chatbot/finetuned_chatbot_gpt.tar.gz > models/finetuned_chatbot_gpt.tar.gz && \
cd models/ && \
tar -xvzf finetuned_chatbot_gpt.tar.gz && \
rm finetuned_chatbot_gpt.tar.gz
CMD ["bash"]
我 运行 到目前为止的步骤：
git clone https://github.com/huggingface/transfer-learning-conv-ai
cd transfer-learning-conv-ai
pip install -r requirements.txt
python -m spacy download en
docker build -t convai .
您是否尝试将 numpy 添加到 requirements.txt？在我看来它不见了。
pip 似乎没有安装预制轮子，而是尝试从源代码构建 spacy。这是一个脆弱的过程，需要额外的依赖。
为避免这种情况，您应该确保 Python 软件包 pip、wheel 和 setuptools 在继续安装之前是最新的。
# replace RUN pip3 install -r /tmp/requirements.txt
RUN python3 -m pip install --upgrade pip setuptools wheel
RUN python3 -m pip install -r /tmp/requirements.txt