从 Gitlab Pipelines 上传到 pypi

Question

我正在尝试使用 Gitlab CI 作业将包上传到 pypi，但我无法让它工作：/有人有工作示例吗？

到目前为止我在我的 .gitlab-ci.yaml 中尝试过的（从我的本地机器上所有这些都在工作）：

与 .pypirc 文件缠绕

- echo "[distutils]" >> ~/.pypirc
- echo "index-servers =" >> ~/.pypirc
- echo "    pypi" >> ~/.pypirc
- echo "" >> ~/.pypirc
- echo "[pypi]" >> ~/.pypirc
- 'echo "repository: https://upload.pypi.org/legacy/" >> ~/.pypirc'
- 'echo "username: ${PYPI_USER}" >> ~/.pypirc'
- 'echo "password: ${PYPI_PASSWORD}" >> ~/.pypirc'
- python3 setup.py check sdist bdist  # This will fail if your creds are bad.
- cat ~/.pypirc
- twine upload dist/* --config-file ~/.pypirc

与之前相同，但 $VARIABLE

[...]
- 'echo "username: $PYPI_USER" >> ~/.pypirc'
- 'echo "password: $PYPI_PASSWORD" >> ~/.pypirc'
[...]

前两个选项，但使用python setup.py ... upload
twine upload dist/* -u $PYPI_USER -p $PYPI_PASSWORD
twine upload dist/* 使用 TWINE_USERNAME 和 TWINE_PASSWORD 环境变量。

...并且总是得到 403 Client Error: Invalid or non-existent authentication information。我运行没有选择...

Answer 1

我使用你的代码的修改版本使这个工作正常：

pypi:
  stage: upload
  script:
  - pip install twine
  - rm -rf dist
  - echo "[distutils]" >> ~/.pypirc
  - echo "index-servers =" >> ~/.pypirc
  - echo "    nexus" >> ~/.pypirc
  - echo "" >> ~/.pypirc
  - echo "[nexus]" >> ~/.pypirc
  - echo "${PYPI_REPO}" >> ~/.pypirc
  - echo "${PYPI_USER}" >> ~/.pypirc
  - echo "${PYPI_PASSWORD}" >> ~/.pypirc
  - python3 setup.py check sdist bdist  # This will fail if your creds are bad.
  - python setup.py sdist bdist_wheel
  - twine upload -r nexus dist/*.tar.gz

区别在于我没有使用“'”并且去掉了 yaml 中的冒号；相反，我将秘密的值设置为 username: myuser

Answer 2

您也可以考虑使用 dpl：我是这样做的：

pip:
  stage: upload
  script:
    - apt-get update -qy
    - apt-get install -y ruby-dev
    - gem install dpl
    - python setup.py sdist
    - dpl --provider=pypi --user=$PIP_USERNAME --password=$PIP_PASSWORD --skip_existing=true
  only:
  - master

您可以在项目的变量部分设置 $PIP_USERNAME 和 $PIP_PASSWORD：settings -> CI/CD -> Variables

Answer 3

我只是使用 TWINE_USERNAME 和 TWINE_PASSWORD 变量，它开箱即用。

这是我 gitlab-ci.yml 中的相关部分（将图像替换为您想要的图像，当然还可以根据您的需要更改所有其他内容，例如 stage、cache 等):

pypi:
    image: docker.km3net.de/base/python:3
    stage: release
    cache: {}
    script:
        - pip install -U twine
        - python setup.py sdist
        - twine upload dist/*
    only:
        - tags

并在GitLab Settings->CI/CD->Variables (https://your-gitlab-instance.oerg/GIT_NAMESPACE/GIT_PROJECT/settings/ci_cd)下添加环境变量:

这是成功的管道：

Answer 4

如果 EOF 出现问题，请确保将 Settings/Repository/Tags 更改为受保护，以便它们再次工作。我发布了 here 更完整的描述。

Answer 5

你也可以在一行中上传python包到私有Pypi服务器（我用的是gilab-ci）：

通过Gitlab CI设置设置环境变量PYPI_SERVER、PYPI_USER和PYPI_PASSWORD

通话

twine upload --repository-url ${PYPI_SERVER} --username $PYPI_USER --password $PYPI_PASSWORDD $artifact

注意： 我必须使用 PIP (pip3 install twine) 中的 twine 而不是我的 Ubuntu 包twine 的版本 10 似乎有一个错误 (zipfile.BadZipFile: File is not a zip file)。

Answer 6

请注意，GitLab 12.10（2020 年 4 月）将提供其高级版或更高版本，一种更简单的方法，使用 CI_JOB_TOKEN（请参阅本答案的第二部分，GitLab 13.4，2020 年 9 月）

Build, publish, and share Python packages to the GitLab PyPI Repository

Python developers need a mechanism to create, share, and consume packages that contain compiled code and other content in projects that use these packages. PyPI, an open source project maintained by the Python Packaging Authority, is the standard for how to define, create, host, and consume Python packages.

In GitLab 12.10, we are proud to offer PyPI repositories built directly into GitLab! Developers now have an easier way to publish their projects’ Python packages. By integrating with PyPI, GitLab will provide a centralized location to store and view those packages in the same place as their source code and pipelines.

In March, we announced that the GitLab PyPI Repository and support for other package manager formats will be moved to open source.
You can follow along as we work to make these features more broadly available in the epic.

参见 Documentation and Issue。

以及 GitLab 13.4（2020 年 9 月）

Use CI_JOB_TOKEN to publish PyPI packages

You can use the GitLab PyPI Repository to build, publish, and share python packages, right alongside your source code and CI/CD Pipelines.

However, previously you couldn’t authenticate with the repository by using the pre-defined environment variable CI_JOB_TOKEN.
As a result, you were forced to use your personal credentials for making updates to the PyPI Repository, or you may have decided not to use the repository at all.

Now it is easier than ever to use GitLab CI/CD to publish and install PyPI packages by using the predefined CI_JOB_TOKEN environment variable.

See Documentation and Issue.

Answer 7

我知道这是一个老问题，但如果您使用的是 poetry（我正在使用版本 1.1.11 进行测试），您可以很容易地做到这一点，如下所示：

poetry config repositories.my_private_repo [URL_TO_YOUR_PYPI_REPO]
poetry config http-basic.my_private_repo [USERNAME] [PASSWORD]
poetry build
poetry publish --repository my_private_repo

在 develop 分支上，您可以将 --dry-run 参数添加到 poetry publish，这样它实际上就不会被上传了

从 Gitlab Pipelines 上传到 pypi

Upload to pypi from Gitlab Pipelines

python

pypi

gitlab-ci

twine

Build, publish, and share Python packages to the GitLab PyPI Repository

Use CI_JOB_TOKEN to publish PyPI packages