python 需求：如何获取相同的版本？

Question

这个问题是校长的结果需求规范的解决方案：

我尝试在服务器之间同步 python 要求和本地开发系统。两者都是 Ubuntu 16.04., 所以这应该是个问题。服务器给了我一个需求文件（在其他问题中描述的 pip-installing django 之后）。

但是在本地机器上 pip-installing 要求会出现一些错误，像这样：

Could not find a version that satisfies the requirement python-    
apt==1.1.0b1 (from -r requirements.txt (line 3)) (from versions: 0.0.0, 
0.7.8) No matching distribution found for python-apt==1.1.0b1 (from -r 
requirements.txt

有没有办法确保可以在类似系统上满足要求没有得到由版本不兼容引起的此类错误（或者有选择地解决不兼容问题）？

我需要虚拟环境来解决这个问题吗？甚至是 pipenv？或者有没有一种简单直接的方法来拥有两个系统兼容 python 和包环境？

Answer 1

的确，python-apt的最新版本是0.7.8 https://pypi.org/project/python-apt/

如果您 100% 确定它是同一个包，请尝试在您的 requirements.txt 文件中更改它。

另一方面，您可以尝试在本地查看 python-apt 在哪里。

import apt
print(apt.__file__)  # or print(apt.__path__)

然后，去那里手动检查 python-apt 包裹到底是什么。

Answer 2

问题

这个特定模块 (python-apt) 是 only available on PyPi with version 0.7.8。然而，这个版本似乎是一个错误！

python-apt has stated the following 的开发者和 Debian 软件包维护者之一：

Aargh, not this whole PyPI thing again. Nobody ever officially uploaded python-apt there. It is tightly coupled with APT, and not supposed to be distributed in any fashion other than via Debian packages.

There is no, and has never been any support for PyPi. And I can say that I have absolutely no interest in duplicating work there. Source: Debian "Deity" Mailing List 2016-11-22 msg#00094

You can install python-apt from apt, we do not provide python-apt on pip. I recently got control over the pypi entry and need to do something with it. I'm not keen on providing python-apt outside of the distro, though (python-apt and apt versions x.y need to match), so I'd rather just get rid of it, so people stop with questions about outdated versions. Source: python-apt#1883451

所以至少对于这个依赖项，看起来我们在通过 pip + PyPi 本地解析依赖项 python 时运气不佳。幸运的是，上游项目托管在 salsa.debian.org GitLab 实例上，并且 pip 现在支持 git+ SCM url，以及其他选项。

解决方案：

一般来说，解决这种依赖的方案有很多。您要解决的问题是：

这个包是从哪里提供的？
- 系统OS包管理器
- 任意上游释放URL
- 开发 SCM 回购
- 分叉 Git 修复错误的回购协议
- 等...
您要安装什么版本？
- 兼容性问题：
  - 必须是>= 2.0.0
  - 带有一些错误修复的分叉回购/功能分支
  - API兼容性：不大于2.x/Semantic Versioning
- 发展：
  - 只使用最新和最好的（前沿）
  - 为我的平台或系统使用特定版本（例如：系统 OS 包，我在开发目录中的本地分叉版本）
- 质量保证/测试：
  - 针对特定版本进行测试
  - 针对最新的测试（例如：夜间构建）
  - 针对系统OS 发布
依赖应该如何解决？
- 对 pip-module-name + version constraint 的“抽象”依赖
  - 允许稍后使用 where 收集（URL / PyPi / Artifactory）和 what版本允许满足约束。
  - 如果需要，用户始终可以通过为 pip 指定参数、使用特定的 virtualenv 安装等来覆盖这些...
- 对特定 URL + 包名 + 版本的“具体”依赖
  - 极端情况：锁定到特定 URL + 版本 + sha1 / sha256 / sha** 并进行校验和验证以确保准确位置和文件完整性。
  - 不太灵活，但最确定的是锁定到一个准确和精确的版本和来源。
您是在开发“应用程序”还是“库”/Python 模块？
- 是否需要使用 setup.py install_requires = [...] 样式解析通过 pip 安装依赖项？（图书馆）
- 应用程序安装程序是否需要通过 pip install -r requirements.txt 安装依赖项？ (申请)
您的项目将如何发布？
- 谁将安装软件包，他们的系统将如何解决依赖关系？
- 它是作为库在 PyPi 上发布，还是作为应用程序在其他地方发布？（遵循一些经验法则）
  - 一般来说：
    - 图书馆往往希望拥有广泛开放的版本说明符
    - 一个应用程序需要非常具体的依赖关系来确保稳定性（很多依赖关系意味着很多通常未经测试的版本排列！）
  - 使用setup.py指定PyPi的依赖库
  - 使用requirements.txt指定应用程序的依赖项
- 这是否要打包为本机 OS 包？（例如：.deb、.rpm、.apk 等...）
  - 本机包管理器也有依赖解析...也许使用它来确保本机兼容性！
- 您的软件包将支持哪些其他 OS 平台以及这些平台将如何解决依赖关系？

因此，大多数情况下，我们可以看到存在各种问题，这些问题都与 所需的特异性 有关，这些依赖项在何处以及如何获得已解决并安装。这里没有“一刀切”的解决方案……有利有弊，只有许多不同的解决方案落在以下范围内：

more specific <---------------------> less specific
reliable compatibility                reliable installability
less testing permutations             more (possibly un-vetted) testing permutations
limited platform support              more platform support (when more permutations are well tested)
dependable known configurations       less dependable known configurations                        
less platform tolerant                more tolerant and agnostic of platforms
more OS native                        less OS native

来自 GitLab Upstream Repo 的最新和最重要

python-apt 包问题的一个解决方案是使用 requirements.txt 中的 git+ URL 功能。这对于针对来自 GitLab 的 python-apt 上游版本的开发非常有用。为了进一步将安装与系统 OS 提供的 python-apt 版本隔离，可能需要 virtualenv 或 pip install --user。例如：

requirements.txt:

--index-url https://pypi.python.org/simple/

-e git+https://salsa.debian.org/apt-team/python-apt.git#egg=python-apt
-e .

这可以被示例项目使用 setup.py 包含：

[...SNIP...] # Boilerplate stuff here

setup(

#[...SNIP...] # Other setup() args here

    platforms=['linux'],
    # Reference:
    #  - https://github.com/pypa/interoperability-peps/pull/30/files#r184839487
    # sudo apt install python3-apt apt-rdepends apt
    # os_requires=[
    #     ['python3-apt', type='packagename', target='run', os='ubuntu'],
    #     ['apt-rdepends', type='packagename', target='run', os='ubuntu'],
    #     ['apt', type='packagename', target='run', os='ubuntu']
    #     ['libapt-pkg-dev', type='packagename', target='build', os='ubuntu']
    # ]
    # Build-deps for apt-python via git SCM: sudo apt install libapt-pkg-dev 
    python_requires='>=3.5',
    install_requires=[
        'python-apt (>= 2.0)',
        # rest of your dependencies here
        #[... SNIP ...]
    ],
    package_dir={'': 'lib'},
    scripts=_glob('bin/*'),

#[...SNIP...]

)

注：os_requires isn't actually supported yet, but is proposed for a PEP。这可能有助于将来对包的外部依赖。在这种情况下，python 模块不是通过 PyPi / pip 分发，而是仅通过 [=331 上的 apt / .deb 包提供，这会有所帮助=].

设置您的 virtualenv 或根据需要使用 pip3 install --user，然后继续。当运行 pip3 install -r requirements.txt:

时，使用 git+ 要求功能会产生以下结果

$ pip3 install -r requirements.txt
Looking in indexes: https://pypi.python.org/simple/
Obtaining file:///../example-project (from -r requirements.txt (line 4))
Obtaining python-apt from git+https://salsa.debian.org/apt-team/python-apt.git#egg=python-apt (from -r requirements.txt (line 3))
  Updating ./example-project-venv/src/python-apt clone
  Running command git fetch -q --tags
  Running command git reset --hard -q c97d4159beae2f9cd42d55d3dff9c37f5c69aa44
ERROR: example-project 0.0.1 has requirement python-apt>=2.0, but you'll have python-apt 0.0.0 which is incompatible.
Installing collected packages: python-apt, example-project
  Running setup.py develop for python-apt
  Running setup.py develop for example-project
Successfully installed example-project python-apt

注意：您可能需要先为 python-apt 安装运行时和构建/setup.py 依赖项：

# Runtime deps (e.g.: Ubuntu 20.04 needs python3-apt, <20.04 needs python-apt):
sudo apt install python3-apt apt
# python-apt pip install deps (also for setup.py / development)
sudo apt install libapt-pkg-dev

备选方案：`dependency_links`（注意：可能已弃用）

如果您正在开发库类型模块，并且还希望使用 GitLab 作为 python-apt 的源代码，您可能需要考虑 using dependency_links in setup.py to provide the git+ or http(s) tarball release URL rather than in requirements.txt. This is helpful to distinguish an "application" python project from a "library" Python module project. It all depends on what your project's install process looks like. (e.g.: do you want to pip install -r requirements.txt, or just pip install example-module, or python[3] setup.py {sdist,bdist,bdist_rpm, etc...}. It could also be helpful to specify a custom URL for a forked version of python-apt. However, this method is likely to be deprecated sometime soon (if not partially already in new versions of pip). You may want to consider other options for future-proofing your dependency specifications such as PEP 508 或 pip install --find-links ... 代替。

此外，"application" vs "library" distinction 在这里可能很重要，“抽象”与“具体”依赖关系的概念也是如此。快速总结可能是：

This split between abstract and concrete is an important one. It was what allows the PyPI mirroring infrastructure to work. It is what allows a company to host their own private package index. It is even what enables you to fork a library to fix a bug or add a feature and use your own fork. Because an abstract dependency is a name and an optional version specifier you can install it from PyPI or from Crate.io, or from your own filesystem. You can fork a library, change the code, and as long as it has the right name and version specifier that library will happily go on using it.

Setuptools has a feature similar to the Go example. It’s called dependency links and it looks like this:
setup(
    # ...
    dependency_links = [
        "http://packages.example.com/snapshots/",
        "http://example2.com/p/bar-1.0.tar.gz",
    ], ) ```

This “feature” of setuptools removes the abstractness of its
dependencies and hardcodes an exact url from which you can fetch the
dependency from. Now very similarly to Go if we want to modify
packages, or simply fetch them from a different server we’ll need to
go in and edit each package in the dependency chain in order to update
the dependency_links.
Source: caremad.io Blog Post: setup.py vs requirements.txt

对于这个 python-apt 示例，我们可能会使用类似这样的东西来锁定 v2.0.0 上的“具体依赖项”：


setup(
    # [...SNIP...]
    dependency_links = [
        "https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz#egg=python-apt"
    ],
    # [...SNIP...]
) `

注意：此“错误功能”是 briefly removed, and then brought back given some usefulness 在指定私有包依赖性 URL 时。但是，目前 pip --process-dependency-links 标志已被弃用，因此它的用处可能仅限于旧版本的 Python 2 + pip.

选择：PEP 508 Syntax

Newer versions of pip now have URL support for PEP 508 syntax. This is probably the most future-proof method for specifying concrete and abstract dependencies with a complex grammar (See PEP 508 了解详情）。现在可以通过多种方式指定包，包括自定义 URLs.

例如，使用可选的 sha256 校验和将 python-apt 锁定为 v2.0.0：

setup(
    # [...SNIP...]
    install_requires=[
        'python-apt@https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz#sha256=1ddbd3eb7cbc1ded7e0e8a2dd75219f0c59c7e062c6e6bfd5c8ff6f656c59a4e',
        # [...SNIP...]
    ],
    # [...SNIP...]
)

requirements.txt:

--index-url https://pypi.python.org/simple/

-e .

然后，pip install -r requirements.txt 在没有任何额外标志的情况下仍然有效：

$ ./example-project-venv/bin/python3 ./example-project-venv/bin/pip3 install -r requirements.txt 
Looking in indexes: https://pypi.python.org/simple/
Obtaining file://./src/pub/example-project (from -r requirements.txt (line 4))
Requirement already satisfied: graph-tools>=1.5 in ./example-project-venv/lib/python3.8/site-packages (from example-project==0.0.1->-r requirements.txt (line 4)) (1.5)
Collecting python-apt@ https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz#sha256=1ddbd3eb7cbc1ded7e0e8a2dd75219f0c59c7e062c6e6bfd5c8ff6f656c59a4e
  Using cached https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz (458 kB)
Building wheels for collected packages: python-apt
  Building wheel for python-apt (setup.py) ... done
  Created wheel for python-apt: filename=python_apt-0.0.0-cp38-cp38-linux_x86_64.whl size=2040980 sha256=79eeb0d1bb9e3c9785acb68f164a3f72a5777539137d180e9ded7558d2547a49
  Stored in directory: ~/.cache/pip/wheels/c4/09/b5/36fc8c9a1ebe8786620db922f1495da200dce187ee7c618993
Successfully built python-apt
Installing collected packages: python-apt, example-project
  Attempting uninstall: example-project
    Found existing installation: example-project 0.0.1
    Uninstalling example-project-0.0.1:
      Successfully uninstalled example-project-0.0.1
  Running setup.py develop for example-project
Successfully installed example-project python-apt-0.0.0

选择：`pip install --find-links ...`

安装锁定到特定版本的“具体依赖项”的另一种替代方法是将 --find-links 传递给 pip install 和已发布的 tarball 文件。此方法可能有助于在给定发行版 URL 的情况下显式安装特定版本。例如，使用 python-apt v2.0.0:

$ ./example-project-venv/bin/python3 ./example-project-venv/bin/pip3 install --find-links 'https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz' -r requirements.txt 
Looking in indexes: https://pypi.python.org/simple/
Looking in links: https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz
Obtaining file://./example-project (from -r requirements.txt (line 4))
Requirement already satisfied: graph-tools>=1.5 in ./example-project-venv/lib/python3.8/site-packages (from example-project==0.0.1->-r requirements.txt (line 4)) (1.5)
Collecting python-apt>=2.0
  Downloading https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz (458 kB)
     |████████████████████████████████| 458 kB 614 kB/s 
  WARNING: Requested python-apt>=2.0 from https://salsa.debian.org/apt-team/python-apt/-/archive/2.0.0/python-apt-2.0.0.tar.gz (from example-project==0.0.1->-r requirements.txt (line 4)), but installing version 0.0.0
Building wheels for collected packages: python-apt
  Building wheel for python-apt (setup.py) ... done
  Created wheel for python-apt: filename=python_apt-0.0.0-cp38-cp38-linux_x86_64.whl size=2040783 sha256=d0a8f88c04f202e948b9855837140517d9b2bd3cef72e626221614552a476780
  Stored in directory: ~/.cache/pip/wheels/8a/07/e9/b3c3328bac08c030a5b1e754e01e327b62fd26f9baedf07c15
Successfully built python-apt
ERROR: example-project 0.0.1 has requirement python-apt>=2.0, but you'll have python-apt 0.0.0 which is incompatible.
Installing collected packages: python-apt, example-project
  Attempting uninstall: python-apt
    Found existing installation: python-apt 0.0.0
    Uninstalling python-apt-0.0.0:
      Successfully uninstalled python-apt-0.0.0
  Attempting uninstall: example-project
    Found existing installation: example-project 0.0.1
    Uninstalling example-project-0.0.1:
      Successfully uninstalled example-project-0.0.1
  Running setup.py develop for example-project
Successfully installed example-project python-apt-0.0.0

基本系统 Debian 软件包

在 Debian 和 Ubuntu 上，您会在各种发行版中看到两个 .deb 软件包：python3-apt 和 python-apt（对于 Python2).

These packages are managed 由 APT 包管理器安装，因此安装在系统位置：/usr/lib/python3/dist-packages 或 /usr/lib/python2.7/dist-packages for Python3 & Python2.7分别。

这个dist-packages路径，和其他Python打包约定是explained well in this post:

The system has installed Python packages in the global dist-packages directory of each Python version and created symbolic links:
/usr/lib/python2.7/dist-packages/numpy

/usr/lib/python3/dist-packages/numpy

ls -ls /usr/include/numpy
#-> ../lib/python2.7/dist-packages/numpy/core/include/numpy

ls -l /usr/include/python2.7/numpy
#->../../lib/python2.7/dist-packages/numpy/core/include/numpy

ls -l /usr/include/python3.5/numpy
#-> ../../lib/python3/dist-packages/numpy/core/include/numpy
Note the good use of dist-packages instead of site-packages which should be reserved for the system Python.

因此，如果您希望使用 python3-apt 的基本 OS 系统级版本，那么您需要确保此路径在您的 sys.path 上或 PYTHONPATH 这样 import apt 就可以了。然而，如果您想使用 site-packages 位置或 virtualenv 位置...这些位置必须存在于 sys.path / PYTHONPATH 上。

不幸的是，as mentioned before，目前还没有官方的方式来声明对 OS 包的依赖，提供特定版本的 python 模块。但是，只要您管理 python 运行时环境的 import 路径，您就应该能够使用位于 dist-packages目录。

python 需求：如何获取相同的版本？

python requirements: how to get identical versions?

python

virtualenv

pipenv

问题