GitLab runner,推送时出错,而同样的事情在控制台上的 git bash 或 python 上起作用
GitLab runner, error on push whereas same thing works on git bash or python from a console
上下文
我们正在尝试做一个 GitLab 运行ner 作业,在某个标记上修改版本头文件并向此变更集添加一个版本 branch/tag。
GitLab 运行ner 服务器在我的机器上,由我的用户作为服务启动(已正确注册到我们的 GitLab 服务器)。
GitLab 运行ner 作业基本上启动了一个 python 脚本,该脚本使用 gitpython 来完成作业,只有几个运行ner yml 文件中的更改(添加 before_script
部分以获得上传权限,从那里获得:),这里是完整的 .gitlab-ci.yml
文件:
variables:
GIT_SUBMODULE_STRATEGY: recursive
stages: [ build, publish, release ]
release_tag:
stage: build
before_script:
- git config --global user.name ${GITLAB_USER_NAME}
- git config --global user.email ${GITLAB_USER_EMAIL}
script:
- python .\scripts\release_gitlab_runner.py
only:
# Trigger on specific regex...
- /^Src_V[0-9]+\.[0-9]+\.[0-9]+$/
except:
# .. only for tags then except branches, see doc (https://docs.gitlab.com/ee/ci/yaml/#regular-expressions): "Only the tag or branch name can be matched by a regular expression."
- branches
还在 python URL 推送时添加技巧(使用 user:personal_access_token@repo_URL
推送而不是默认 运行ner URL,从相同的答案中获得如上所述,令牌已从公司 gitlab => 用户 "Settings" => "Access Tokens" => "Add a personal access token" 生成,拥有所有权利且永不过期),这里是,不是实际的 scripts\release_gitlab_runner.py
python 脚本,而是一个简化的 git 流程,尽可能符合我们想要的标准(获取所有内容,创建具有随机名称的本地分支,这样它就不会存在、修改文件、暂存、提交并最终推送):
# -*-coding:utf-8 -*
import uuid
import git
import sys
import os
# Since we are in <git root path>/scripts folder, git root path is this file's path parent path
GIT_ROOT_PATH = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
try:
# Get user login and URL from GITLAB_USER_LOGIN and CI_REPOSITORY_URL gitlab environment variables
gitlabUserLogin = os.environ["GITLAB_USER_LOGIN"]
gitlabFullURL = os.environ["CI_REPOSITORY_URL"]
# Push at "https://${GITLAB_USER_NAME}:${PERSONAL_ACCESS_TOKEN}@gitlab.companyname.net/demo/demo.git")
# generatedPersonalAccessToken has been generated with full rights from https://gitlab.companyname.net/profile/personal_access_tokens and set in a variable not seen here
gitlabPushURL = "https://{}:{}@{}".format(gitlabUserLogin, generatedPersonalAccessToken, gitlabFullURL.split("@")[-1])
print("gitlabFullURL is [{}]".format(gitlabFullURL))
print("gitlabPushURL is [{}]".format(gitlabPushURL))
branchName = str(uuid.uuid1())
print("Build git.Repo object with [{}] root path".format(GIT_ROOT_PATH))
repo = git.Repo(GIT_ROOT_PATH)
print("Fetch all")
repo.git.fetch("-a")
print("Create new local branch [{}]".format(branchName))
repo.git.checkout("-b", branchName)
print("Modify file")
versionFile = os.path.join(GIT_ROOT_PATH, "public", "include" , "Version.h")
patchedVersionFileContent = ""
with open(versionFile, 'r') as versionFileContent:
patchedVersionFileContent = versionFileContent.read()
patchedVersionFileContent = re.sub("#define VERSION_MAJOR 0", "#define VERSION_MAJOR {}".format(75145), patchedVersionFileContent)
with open(versionFile, 'w') as versionFileContent:
versionFileContent.write(patchedVersionFileContent)
print("Stage file")
repo.git.add("-u")
print("Commit file")
repo.git.commit("-m", "New version file in new branch {}".format(branchName))
print("Push new branch [{}] remotely".format(branchName))
# The error is at below line:
repo.git.push(gitlabPushURL, "origin", branchName)
sys.exit(0)
except Exception as e:
print("Exception: {}".format(e))
sys.exit(-1)
问题
即使使用了拥有权限的技巧,当我们尝试从 GitLab 运行ner 推送时,也会出现以下错误:
Cmd('git') failed due to: exit code(1)
cmdline: git push https://user:token@gitlab.companyname.net/demo/repo.git origin 85a3fa6e-690a-11ea-a07d-e454e8696d31
stderr: 'error: src refspec origin does not match any
error: failed to push some refs to 'https://user:token@gitlab.companyname.net/demo/repo.git''
什么有效
如果我打开 Git Bash,我成功 运行 手动命令:
git fetch -a
git checkout -b newBranch
vim public/include/Version.h
=> At this point file has been modified
git add -u
git commit -m "New version file in new branch"
git push origin newBranch
在这里,如果我们从其他地方获取所有内容,我们可以看到带有版本文件修改的 newBranch
同样,如果我们从 python 命令行 运行 脚本内容(没有 URL 修改)(假设已执行脚本中的所有导入):
GIT_ROOT_PATH = "E:\path\to\workspace\repo"
branchName = str(uuid.uuid1())
repo = git.Repo(GIT_ROOT_PATH)
repo.git.fetch("-a")
repo.git.checkout("-b", branchName)
versionFile = os.path.join(GIT_ROOT_PATH, "public", "include" , "Version.h")
patchedVersionFileContent = ""
with open(versionFile, 'r') as versionFileContent:
patchedVersionFileContent = versionFileContent.read()
patchedVersionFileContent = re.sub("#define VERSION_MAJOR 0", "#define VERSION_MAJOR {}".format(75145), patchedVersionFileContent)
with open(versionFile, 'w') as versionFileContent:
versionFileContent.write(patchedVersionFileContent)
repo.git.add("-u")
repo.git.commit("-m", "New version file in new branch {}".format(branchName))
repo.git.push("origin", branchName)
结论
当从 GitLab 运行 中 运行ning 时,我找不到我做错了什么,是不是我遗漏了什么?
当从 GitLab 运行ner 运行ning 时,我唯一能看到的不同是,在获取之后,我可以看到我在一个分离的头上(列表 repo.git.branch('-a').split('\n')
例如给出 ['* (HEAD detached at 560976b)', 'branchName', 'remotes/origin/otherExistingBranch', ...]),但这应该不是问题,因为我创建了一个新分支推动,对吗?
Git 说你用错了refspec。当您需要推入其他遥控器时,您必须先 gitlab = repo.create_remote("gitlab", gitlabPushURL)
并像 repo.push("gitlab", branchName)
.
一样推入它
从@gluttony 编辑到下一个 git 运行 与 "remote already exists":
不中断
remote_name = "gitlab"
if remote_name not in repo.remotes:
repo.create_remote(remote_name, gitlabPushURL)
上下文
我们正在尝试做一个 GitLab 运行ner 作业,在某个标记上修改版本头文件并向此变更集添加一个版本 branch/tag。
GitLab 运行ner 服务器在我的机器上,由我的用户作为服务启动(已正确注册到我们的 GitLab 服务器)。
GitLab 运行ner 作业基本上启动了一个 python 脚本,该脚本使用 gitpython 来完成作业,只有几个运行ner yml 文件中的更改(添加 before_script
部分以获得上传权限,从那里获得:.gitlab-ci.yml
文件:
variables:
GIT_SUBMODULE_STRATEGY: recursive
stages: [ build, publish, release ]
release_tag:
stage: build
before_script:
- git config --global user.name ${GITLAB_USER_NAME}
- git config --global user.email ${GITLAB_USER_EMAIL}
script:
- python .\scripts\release_gitlab_runner.py
only:
# Trigger on specific regex...
- /^Src_V[0-9]+\.[0-9]+\.[0-9]+$/
except:
# .. only for tags then except branches, see doc (https://docs.gitlab.com/ee/ci/yaml/#regular-expressions): "Only the tag or branch name can be matched by a regular expression."
- branches
还在 python URL 推送时添加技巧(使用 user:personal_access_token@repo_URL
推送而不是默认 运行ner URL,从相同的答案中获得如上所述,令牌已从公司 gitlab => 用户 "Settings" => "Access Tokens" => "Add a personal access token" 生成,拥有所有权利且永不过期),这里是,不是实际的 scripts\release_gitlab_runner.py
python 脚本,而是一个简化的 git 流程,尽可能符合我们想要的标准(获取所有内容,创建具有随机名称的本地分支,这样它就不会存在、修改文件、暂存、提交并最终推送):
# -*-coding:utf-8 -*
import uuid
import git
import sys
import os
# Since we are in <git root path>/scripts folder, git root path is this file's path parent path
GIT_ROOT_PATH = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
try:
# Get user login and URL from GITLAB_USER_LOGIN and CI_REPOSITORY_URL gitlab environment variables
gitlabUserLogin = os.environ["GITLAB_USER_LOGIN"]
gitlabFullURL = os.environ["CI_REPOSITORY_URL"]
# Push at "https://${GITLAB_USER_NAME}:${PERSONAL_ACCESS_TOKEN}@gitlab.companyname.net/demo/demo.git")
# generatedPersonalAccessToken has been generated with full rights from https://gitlab.companyname.net/profile/personal_access_tokens and set in a variable not seen here
gitlabPushURL = "https://{}:{}@{}".format(gitlabUserLogin, generatedPersonalAccessToken, gitlabFullURL.split("@")[-1])
print("gitlabFullURL is [{}]".format(gitlabFullURL))
print("gitlabPushURL is [{}]".format(gitlabPushURL))
branchName = str(uuid.uuid1())
print("Build git.Repo object with [{}] root path".format(GIT_ROOT_PATH))
repo = git.Repo(GIT_ROOT_PATH)
print("Fetch all")
repo.git.fetch("-a")
print("Create new local branch [{}]".format(branchName))
repo.git.checkout("-b", branchName)
print("Modify file")
versionFile = os.path.join(GIT_ROOT_PATH, "public", "include" , "Version.h")
patchedVersionFileContent = ""
with open(versionFile, 'r') as versionFileContent:
patchedVersionFileContent = versionFileContent.read()
patchedVersionFileContent = re.sub("#define VERSION_MAJOR 0", "#define VERSION_MAJOR {}".format(75145), patchedVersionFileContent)
with open(versionFile, 'w') as versionFileContent:
versionFileContent.write(patchedVersionFileContent)
print("Stage file")
repo.git.add("-u")
print("Commit file")
repo.git.commit("-m", "New version file in new branch {}".format(branchName))
print("Push new branch [{}] remotely".format(branchName))
# The error is at below line:
repo.git.push(gitlabPushURL, "origin", branchName)
sys.exit(0)
except Exception as e:
print("Exception: {}".format(e))
sys.exit(-1)
问题
即使使用了拥有权限的技巧,当我们尝试从 GitLab 运行ner 推送时,也会出现以下错误:
Cmd('git') failed due to: exit code(1)
cmdline: git push https://user:token@gitlab.companyname.net/demo/repo.git origin 85a3fa6e-690a-11ea-a07d-e454e8696d31
stderr: 'error: src refspec origin does not match any
error: failed to push some refs to 'https://user:token@gitlab.companyname.net/demo/repo.git''
什么有效
如果我打开 Git Bash,我成功 运行 手动命令:
git fetch -a
git checkout -b newBranch
vim public/include/Version.h
=> At this point file has been modified
git add -u
git commit -m "New version file in new branch"
git push origin newBranch
在这里,如果我们从其他地方获取所有内容,我们可以看到带有版本文件修改的 newBranch
同样,如果我们从 python 命令行 运行 脚本内容(没有 URL 修改)(假设已执行脚本中的所有导入):
GIT_ROOT_PATH = "E:\path\to\workspace\repo"
branchName = str(uuid.uuid1())
repo = git.Repo(GIT_ROOT_PATH)
repo.git.fetch("-a")
repo.git.checkout("-b", branchName)
versionFile = os.path.join(GIT_ROOT_PATH, "public", "include" , "Version.h")
patchedVersionFileContent = ""
with open(versionFile, 'r') as versionFileContent:
patchedVersionFileContent = versionFileContent.read()
patchedVersionFileContent = re.sub("#define VERSION_MAJOR 0", "#define VERSION_MAJOR {}".format(75145), patchedVersionFileContent)
with open(versionFile, 'w') as versionFileContent:
versionFileContent.write(patchedVersionFileContent)
repo.git.add("-u")
repo.git.commit("-m", "New version file in new branch {}".format(branchName))
repo.git.push("origin", branchName)
结论
当从 GitLab 运行 中 运行ning 时,我找不到我做错了什么,是不是我遗漏了什么?
当从 GitLab 运行ner 运行ning 时,我唯一能看到的不同是,在获取之后,我可以看到我在一个分离的头上(列表 repo.git.branch('-a').split('\n')
例如给出 ['* (HEAD detached at 560976b)', 'branchName', 'remotes/origin/otherExistingBranch', ...]),但这应该不是问题,因为我创建了一个新分支推动,对吗?
Git 说你用错了refspec。当您需要推入其他遥控器时,您必须先 gitlab = repo.create_remote("gitlab", gitlabPushURL)
并像 repo.push("gitlab", branchName)
.
从@gluttony 编辑到下一个 git 运行 与 "remote already exists":
不中断remote_name = "gitlab"
if remote_name not in repo.remotes:
repo.create_remote(remote_name, gitlabPushURL)