詹金斯获取太慢
Jenkins fetch is too slow
我正在为我的公司配置 Jenkins 环境。
我们使用 gitea (1.9.3)、Jenkins 2.194 和 git 版本 2.23
当我从源代码管理构建存储库时，它会构建存储库，但执行获取命令需要 120-150 秒。
有什么建议吗？
我尝试更改 git 客户端，确保凭据正常，尝试使用空或完整存储库以及子模块和只有一个分支配置。
有问题的部分：
C:\Program Files\Git\cmd\git.exe fetch --no-tags --force --progress https://XXX:3000/Tests/test-repo.git +refs/heads/:refs/remotes/origin/ *
编辑 1：构建需要 4:27 无论存储库是空的还是满的东西，如果我 运行 手动从 git 获取命令需要大约 3 秒服务器。
Git 相当快，但如果您的存储库非常大，下载整个历史记录可能需要一些时间。
一种选择是将 --depth 1 参数用于 git clone 或 git fetch。这个参数告诉 git 只下载最近的提交，而不是完整的历史记录。您将无法在提交之间切换、查看差异等，但由于这是针对构建环境的，因此您不需要任何这些。
我尝试在 Ubuntu 服务器 18.04 上安装 Jenkins，它使用子模块进行完整构建的时间从 4.5 分钟缩短到大约 20 秒。
I tried to install Jenkins on Ubuntu server 18.04 and it shorten the time from 4.5 minutes to around 20 seconds for a full build with sub modules.
Windows 上任何 Git 超过 2.20 的性能回归 on the mailing list 的确如此。
也在 git-for-windows/git issue 2199.
使用 Git 2.29（2020 年第 4 季度），围绕子模块处理的优化将使 Windows。
参见 commit 7ea0c2f (04 Sep 2020) by Orgad Shaneh (orgads)。
（由 Junio C Hamano -- gitster -- in commit bcb68bf 合并，2020 年 9 月 22 日）
fetch: do not look for submodule changes in unchanged refs
Signed-off-by: Orgad Shaneh
When fetching recursively with submodules, for each ref in the superproject, we call check_for_new_submodule_commits() which collects all the objects that have to be checked for submodule changes on calculate_changed_submodule_paths().
On the first call, it also collects all the existing refs for excluding them from the scan.
calculate_changed_submodule_paths() creates an argument array with all the collected new objects, followed by --not and all the old objects.
This argv is passed to setup_revisions, which parses each argument, converts it back to an oid and resolves the object.
The parsing itself also does redundant work, because it is treated like user input, while in fact it is a full oid. So it needlessly attempts to look it up as ref (checks if it has ^, ~ etc.), checks if it is a file name etc.
For a repository with many refs, all of this is expensive. But if the fetch in the superproject did not update the ref (i.e. the objects that are required to exist in the submodule did not change), there is no need to include it in the list.
Before commit be76c212 ("fetch: ensure submodule objects fetched", 2018-12-06, Git v2.21.0-rc0 -- merge listed in batch #4), submodule reference changes were only detected for refs that were changed, but not for new refs. This commit covered also this case, but what it did was to just include every ref.
This change should reduce the number of scanned refs by about half (except the case of a no-op fetch, which will not scan any ref), because all the existing refs will still be listed after --not.
我正在为我的公司配置 Jenkins 环境。 我们使用 gitea (1.9.3)、Jenkins 2.194 和 git 版本 2.23 当我从源代码管理构建存储库时，它会构建存储库，但执行获取命令需要 120-150 秒。 有什么建议吗？
我尝试更改 git 客户端，确保凭据正常，尝试使用空或完整存储库以及子模块和只有一个分支配置。
有问题的部分：
C:\Program Files\Git\cmd\git.exe fetch --no-tags --force --progress https://XXX:3000/Tests/test-repo.git +refs/heads/:refs/remotes/origin/ *
编辑 1：构建需要 4:27 无论存储库是空的还是满的东西，如果我 运行 手动从 git 获取命令需要大约 3 秒服务器。
Git 相当快，但如果您的存储库非常大，下载整个历史记录可能需要一些时间。
一种选择是将 --depth 1 参数用于 git clone 或 git fetch。这个参数告诉 git 只下载最近的提交，而不是完整的历史记录。您将无法在提交之间切换、查看差异等，但由于这是针对构建环境的，因此您不需要任何这些。
我尝试在 Ubuntu 服务器 18.04 上安装 Jenkins，它使用子模块进行完整构建的时间从 4.5 分钟缩短到大约 20 秒。
I tried to install Jenkins on Ubuntu server 18.04 and it shorten the time from 4.5 minutes to around 20 seconds for a full build with sub modules.
Windows 上任何 Git 超过 2.20 的性能回归 on the mailing list 的确如此。
也在 git-for-windows/git issue 2199.
使用 Git 2.29（2020 年第 4 季度），围绕子模块处理的优化将使 Windows。
参见 commit 7ea0c2f (04 Sep 2020) by Orgad Shaneh (orgads)。
（由 Junio C Hamano -- gitster -- in commit bcb68bf 合并，2020 年 9 月 22 日）
fetch: do not look for submodule changes in unchanged refsSigned-off-by: Orgad Shaneh
When fetching recursively with submodules, for each ref in the superproject, we call
check_for_new_submodule_commits()which collects all the objects that have to be checked for submodule changes oncalculate_changed_submodule_paths().
On the first call, it also collects all the existing refs for excluding them from the scan.
calculate_changed_submodule_paths()creates an argument array with all the collected new objects, followed by --not and all the old objects. Thisargvis passed tosetup_revisions,which parses each argument, converts it back to an oid and resolves the object.
The parsing itself also does redundant work, because it is treated like user input, while in fact it is a full oid. So it needlessly attempts to look it up as ref (checks if it has^,~etc.), checks if it is a file name etc.For a repository with many refs, all of this is expensive. But if the fetch in the superproject did not update the ref (i.e. the objects that are required to exist in the submodule did not change), there is no need to include it in the list.
Before commit be76c212 ("
fetch: ensure submodule objects fetched", 2018-12-06, Git v2.21.0-rc0 -- merge listed in batch #4), submodule reference changes were only detected for refs that were changed, but not for new refs. This commit covered also this case, but what it did was to just include every ref.This change should reduce the number of scanned refs by about half (except the case of a no-op fetch, which will not scan any ref), because all the existing refs will still be listed after
--not.