在 docker 中的 requirments.txt 文件上获取版本冲突,当我在本地 venv 中 运行 时不会发生

Getting version clashes on requirments.txt file in docker that don't occur when I run in a local venv

我在尝试构建 docker 文件时遇到版本冲突。当我在本地 venv 运行 pip install -r requirements.txt 时不会发生这种情况。我在本地venv 运行 pipdeptree 安装后没有发现冲突。这仅在我尝试通过 eb cli 使用 docker 进行构建时发生。

我是 运行宁 docker-ce 19.03.9。我的本地 venv 和我的 docker 文件都配置为 运行 Python 3.6。我一直无法联系到原始开发人员,而且我对 docker 也不是很熟悉,所以除了安装不同版本的 docker 并重试之外,我不知道从这里可以去哪里。

应要求,这里是dokefile;

FROM       python:3.6

WORKDIR    /var/app

RUN        pip3 install virtualenv
RUN        virtualenv /var/app
RUN        /var/app/bin/pip install --upgrade pip
RUN        /var/app/bin/pip install uwsgi

RUN        useradd uwsgi -s /bin/false
RUN        mkdir /var/log/uwsgi
RUN        chown -R uwsgi:uwsgi /var/log/uwsgi

ADD        ./requirements.txt /var/app
RUN        /var/app/bin/pip install -r /var/app/requirements.txt

ADD        . /var/app

<env stuff for uwsgi>

EXPOSE     8080

ADD        uwsgi-start.sh /

CMD        []
ENTRYPOINT ["/uwsgi-start.sh"]

要求列表是;

appnope==0.1.0
astroid==1.6.1
autopep8==1.3.4
awscli==1.16.46
backcall==0.1.0
bleach==2.1.3
boto3==1.5.34
botocore==1.12.36
Cerberus==0.9.2
certifi==2018.4.16
cffi==1.11.4
chardet==3.0.4
click==6.7
decorator==4.3.0
dnspython==1.15.0
docutils==0.14
dominate==2.3.1
ecdsa==0.13
entrypoints==0.2.3
envs==1.2.4
Eve==0.7.8
Eve-Swagger==0.0.8
Events==0.2.2
Faker==0.8.13
flake8==3.5.0
Flask==0.12.2
Flask-Bootstrap==3.3.7.1
Flask-Cors==3.0.3
flask-nav==0.6
Flask-PyMongo==0.5.1
future==0.16.0
gunicorn==19.7.1
html5lib==1.0.1
httplib2==0.10.3
idna==2.6
ipykernel==4.8.2
ipython==6.3.1
ipython-genutils==0.2.0
ipywidgets==7.2.1
isort==4.3.4
itsdangerous==0.24
jedi==0.12.0
Jinja2==2.10
jmespath==0.9.3
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.3
jupyter-console==5.2.0
jupyter-core==4.4.0
lazy-object-proxy==1.3.1
MarkupSafe==1.1.0
mccabe==0.6.1
mistune==0.8.3
nbconvert==5.3.1
nbformat==4.4.0
notebook==5.4.1
numpy==1.14.2
oauth2client==4.1.2
pandas==0.22.0
pandocfilters==1.4.2
parso==0.2.0
pexpect==4.5.0
pickleshare==0.7.4
pkginfo==1.4.2
prompt-toolkit==1.0.15
ptyprocess==0.5.2
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycodestyle==2.3.1
pycparser==2.18
pycryptodome==3.6.1
pyflakes==1.6.0
Pygments==2.2.0
PyJWT==1.6.1
pylint==1.8.2
pymongo==3.6.0
python-dateutil==2.7.2
python-http-client==3.0.0
python-jose-cryptodome==1.3.2
pytz==2018.4
PyYAML==3.13
pyzmq==17.0.0
qtconsole==4.3.1
readme-renderer==24.0
requests==2.18.4
requests-toolbelt==0.8.0
rsa==3.4.2
s3transfer==0.1.13
Send2Trash==1.5.0
sendgrid==5.3.0
simplegeneric==0.8.1
simplejson==3.13.2
six==1.11.0
stringcase==1.2.0
terminado==0.8.1
testpath==0.3.1
text-unidecode==1.2
tornado==5.0.2
tqdm==4.28.1
traitlets==4.3.2
urllib3==1.22
visitor==0.1.3
warrant==0.6.1
watchtower==0.5.3
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.14.1
widgetsnbextension==3.2.1
wrapt==1.10.11
xlrd==1.1.0
XlsxWriter==1.0.2

第一次冲突发生在 awscli 和 boto3 之间;

ERROR: Cannot install -r /var/app/requirements.txt (line 4), -r /var/app/requirements.txt (line 7) and botocore==1.12.36 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested botocore==1.12.36
    awscli 1.16.46 depends on botocore==1.12.36
    boto3 1.5.34 depends on botocore<1.9.0 and >=1.8.48
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

当我删除版本控制约束时,它会发现 werkzeug 出错。

ERROR: Cannot install -r /var/app/requirements.txt (line 22) and Werkzeug==0.14.1 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested Werkzeug==0.14.1
    eve 0.7.8 depends on werkzeug<=0.11.15 and >=0.9.4
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
The command '/bin/sh -c /var/app/bin/pip install -r /var/app/requirements.txt' returned a non-zero code: 1

随着我尝试修复冲突,越来越多的问题出现。有十几个。我无法更改依赖项要求,因为这可能会破坏应用程序,而且我没有足够的时间来尝试更新这些更改的代码库。

instructions Pip helpfully links you to explain what's going on,确实有点毛毛的情况

There is already an Beanstalk application that's running EC2s with docker containers that work just fine with these requirements so I'm not sure why this is happening now.

至于“为什么现在会这样”——我能想到的两件事:

  • 您的基础镜像中的Pip版本已更新;较新的版本更聪明地解决了相互冲突的依赖关系(因为它们拒绝安装 might/should 不起作用的软件包)。
  • 因为你的requirements.txt不一定完全locked down;文件中未列出的已安装传递依赖项彼此不兼容。

但是,既然您已经 拥有可用的容器映像,那就太好了!你可以简单地做

$ mv requirements.txt requirements.in
$ docker run -it thatcontainerimage /var/app/bin/pip freeze -l > requirements.txt

  • 首先移动您的“模板”要求,例如pip-tools(我最喜欢的依赖锁定工具)到 requirements.in
  • 从容器获取实际安装包的完整列表到“锁定”requirements.txt 文件。

该需求文件应该可以正常安装。如果仍然失败,(错误的)解决方法是在单独的 pip 调用中安装每个需求;然后 Pip 将 警告 关于冲突的依赖项,但不会拒绝安装它们...

顺便说一句,不过:

  • 当您还使用 Eve==0.7.8 时,您确定要在您的要求中明确指定 Werkzeug==0.14.1 吗? (同样代表例如 MarkupSafe 和 Flask 版本。)
  • 您绝对确定您的正式版应用程序图像需要 awscli 吗?或者 autopep8pylint?或者 gunicorn,如果你是 运行 uwsgi 呢?所有那些笔记本相关的包?您应该将您的要求减少到单独的集合,例如开发和测试,以及运行时...