docker 当 pip 要求中包含雪花依赖项时构建失败
docker build is failing when snowflake dependency is included in the pip requirements
我正在尝试从我的 python 应用程序构建 docker 图像。但是当我将 "snowflake-connector-python" 添加到 pip requirements.txt 文件时，docker 构建失败。
requirements.txt:-
boto3
pycryptodome
snowflake-connector-python
Docker文件：-
FROM python:alpine3.6
RUN python --version
COPY . /sample-app
WORKDIR /sample-app
RUN python -m pip install --upgrade pip
RUN pip install --upgrade setuptools
RUN pip --version
RUN pip install -r requirements.txt
CMD [ "python", "runner.py" ]
CLI docker 构建：-
docker build -t sample-app:1.0 .
错误：-
Step 8/9 : RUN pip install -r requirements.txt
---> Running in 13ab5873e0b5
Collecting boto3 (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/92/37/bcd816851f8a04de7d2ba9df5893acc4c6680fb3ab4442895490724143fb/boto3-1.7.67-py2.py3-none-any.whl (128kB)
Collecting pycryptodome (from -r requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/96/15/a2a2aa89f57a17075579fcd296a0fcff98640776666a4371d33f7f613bb9/pycryptodome-3.6.4.tar.gz (7.1MB)
Collecting snowflake-connector-python (from -r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/66/df/e44ec4d55fce341c9756f5f6012049b85f6a679fc3a423eefd7f66527a46/snowflake_connector_python-1.6.5-py2.py3-none-any.whl (142kB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
Collecting botocore<1.11.0,>=1.10.67 (from boto3->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/d6/81/59f34f9965277be8e480efea09b949e406b1afbfcf29a87c6ae6c2b52121/botocore-1.10.67-py2.py3-none-any.whl (4.4MB)
Collecting s3transfer<0.2.0,>=0.1.10 (from boto3->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
Collecting pyjwt (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
Collecting pyasn1<0.5.0,>=0.2.1 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
Collecting six (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting pyOpenSSL<18.0.0,>=16.2.0 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/79/db/7c0cfe4aa8341a5fab4638952520d8db6ab85ff84505e12c00ea311c3516/pyOpenSSL-17.5.0-py2.py3-none-any.whl (53kB)
Collecting cryptography<2.3,>=1.8.2 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/ec/b2/faa78c1ab928d2b2c634c8b41ff1181f0abdd9adf9193211bd606ffa57e2/cryptography-2.2.2.tar.gz (443kB)
Collecting azure-storage<=0.36.0,>0.34.2 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/76/26/482c033f8f3a48d16cf75aad91c3e1256856719f4117fabb4696d33aa5da/azure_storage-0.36.0-py2.py3-none-any.whl (190kB)
Collecting azure-common<2 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/91/66/6636001c0b6be5423f4bbaffe5febbf9eda39bd1941eb39a73504d8f6f71/azure_common-1.1.14-py2.py3-none-any.whl
Collecting ijson (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/7f/e9/8508c5f4987ba238a2b169e582c1f70a47272b22a2f1fb06b9318201bb9e/ijson-2.3-py2.py3-none-any.whl
Collecting cffi>=1.9 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/e7/a7/4cd50e57cc6f436f1cc3a7e8fa700ff9b8b4d471620629074913e3735fb2/cffi-1.11.5.tar.gz (438kB)
Complete output from command python setup.py egg_info:
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.freenode.net.)
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-7dudc372/cffi/
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
可能是什么问题？我在这里遗漏了什么吗？
https://docs.snowflake.net/manuals/user-guide/python-connector-install.html
将编译 libraries/package 添加到您的 alpine 映像中，在您的 Dockerfile RUN 中添加下面的语句 above/before pip install -
WORKDIR /sample-app
RUN apk add --no-cache build-base libffi-dev openssl-dev ncurses-dev
RUN python -m pip install --upgrade pip
这基本上是安装 gcc、g++ 和 make 来编译包，您的情况也需要其他包。我猜它们已被删除或未随原始 python:alpine3.6 图片一起运送。
alpine linux 非常小，他们必须删除所有内容，因此您必须自己安装。一些 pip 包需要 compiled/build 在你的机器上，但 alpine 缺少这些依赖项，所以安装它们，然后执行你的 pip 安装，然后删除它们，因为它们不再需要，你将减小图像大小。
雪花docs还说你必须安装libffi-dev和openssl-dev
FROM python:alpine3.6
COPY . /sample-app
WORKDIR /sample-app
RUN apk add --update --no-cache --virtual build-deps gcc python3-dev musl-dev libc-dev linux-headers libxslt-dev libxml2-dev \
&& apk add libffi-dev openssl-dev \
&& pip install --upgrade pip setuptools \
&& pip install -r requirements.txt \
&& apk del build-deps
CMD [ "python", "runner.py" ]
我发现在 linux 之上使用 virtual env 有助于为 AWS lambda 构建 python 部署包，但不确定这在这里是否有用。
我需要像 Mazel Tov 所建议的那样yum -y install gcc libffi-devel openssl-devel，以获得 pip 中未包含的其余依赖项。
我正在尝试从我的 python 应用程序构建 docker 图像。但是当我将 "snowflake-connector-python" 添加到 pip requirements.txt 文件时，docker 构建失败。
requirements.txt:-
boto3
pycryptodome
snowflake-connector-python
Docker文件：-
FROM python:alpine3.6
RUN python --version
COPY . /sample-app
WORKDIR /sample-app
RUN python -m pip install --upgrade pip
RUN pip install --upgrade setuptools
RUN pip --version
RUN pip install -r requirements.txt
CMD [ "python", "runner.py" ]
CLI docker 构建：-
docker build -t sample-app:1.0 .
错误：-
Step 8/9 : RUN pip install -r requirements.txt
---> Running in 13ab5873e0b5
Collecting boto3 (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/92/37/bcd816851f8a04de7d2ba9df5893acc4c6680fb3ab4442895490724143fb/boto3-1.7.67-py2.py3-none-any.whl (128kB)
Collecting pycryptodome (from -r requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/96/15/a2a2aa89f57a17075579fcd296a0fcff98640776666a4371d33f7f613bb9/pycryptodome-3.6.4.tar.gz (7.1MB)
Collecting snowflake-connector-python (from -r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/66/df/e44ec4d55fce341c9756f5f6012049b85f6a679fc3a423eefd7f66527a46/snowflake_connector_python-1.6.5-py2.py3-none-any.whl (142kB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
Collecting botocore<1.11.0,>=1.10.67 (from boto3->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/d6/81/59f34f9965277be8e480efea09b949e406b1afbfcf29a87c6ae6c2b52121/botocore-1.10.67-py2.py3-none-any.whl (4.4MB)
Collecting s3transfer<0.2.0,>=0.1.10 (from boto3->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
Collecting pyjwt (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
Collecting pyasn1<0.5.0,>=0.2.1 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
Collecting six (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting pyOpenSSL<18.0.0,>=16.2.0 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/79/db/7c0cfe4aa8341a5fab4638952520d8db6ab85ff84505e12c00ea311c3516/pyOpenSSL-17.5.0-py2.py3-none-any.whl (53kB)
Collecting cryptography<2.3,>=1.8.2 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/ec/b2/faa78c1ab928d2b2c634c8b41ff1181f0abdd9adf9193211bd606ffa57e2/cryptography-2.2.2.tar.gz (443kB)
Collecting azure-storage<=0.36.0,>0.34.2 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/76/26/482c033f8f3a48d16cf75aad91c3e1256856719f4117fabb4696d33aa5da/azure_storage-0.36.0-py2.py3-none-any.whl (190kB)
Collecting azure-common<2 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/91/66/6636001c0b6be5423f4bbaffe5febbf9eda39bd1941eb39a73504d8f6f71/azure_common-1.1.14-py2.py3-none-any.whl
Collecting ijson (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/7f/e9/8508c5f4987ba238a2b169e582c1f70a47272b22a2f1fb06b9318201bb9e/ijson-2.3-py2.py3-none-any.whl
Collecting cffi>=1.9 (from snowflake-connector-python->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/e7/a7/4cd50e57cc6f436f1cc3a7e8fa700ff9b8b4d471620629074913e3735fb2/cffi-1.11.5.tar.gz (438kB)
Complete output from command python setup.py egg_info:
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.freenode.net.)
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-7dudc372/cffi/
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
可能是什么问题？我在这里遗漏了什么吗？
https://docs.snowflake.net/manuals/user-guide/python-connector-install.html
将编译 libraries/package 添加到您的 alpine 映像中，在您的 Dockerfile RUN 中添加下面的语句 above/before pip install -
WORKDIR /sample-app
RUN apk add --no-cache build-base libffi-dev openssl-dev ncurses-dev
RUN python -m pip install --upgrade pip
这基本上是安装 gcc、g++ 和 make 来编译包，您的情况也需要其他包。我猜它们已被删除或未随原始 python:alpine3.6 图片一起运送。
alpine linux 非常小，他们必须删除所有内容，因此您必须自己安装。一些 pip 包需要 compiled/build 在你的机器上，但 alpine 缺少这些依赖项，所以安装它们，然后执行你的 pip 安装，然后删除它们，因为它们不再需要，你将减小图像大小。
雪花docs还说你必须安装libffi-dev和openssl-dev
FROM python:alpine3.6
COPY . /sample-app
WORKDIR /sample-app
RUN apk add --update --no-cache --virtual build-deps gcc python3-dev musl-dev libc-dev linux-headers libxslt-dev libxml2-dev \
&& apk add libffi-dev openssl-dev \
&& pip install --upgrade pip setuptools \
&& pip install -r requirements.txt \
&& apk del build-deps
CMD [ "python", "runner.py" ]
我发现在 linux 之上使用 virtual env 有助于为 AWS lambda 构建 python 部署包，但不确定这在这里是否有用。
我需要像 Mazel Tov 所建议的那样yum -y install gcc libffi-devel openssl-devel，以获得 pip 中未包含的其余依赖项。