git 通过慢速网络克隆或拉取一个巨大的 repo 以致命的 EOF 结束
git clone or pull of a huge repo over slow network ends in fatal EOF
我有一个超过 2 GB 的远程仓库（1 个主分支）。我有大约 2-4 MbPS link 的 ADSL 宽带连接。我试图克隆回购协议。
但是我收到以下错误：
fatal: 远端意外挂断
致命的：早期的 EOF
致命：索引包失败
我在 Stack Overflow 和其他地方搜索并尝试克隆一个裸仓库。
后来我尝试增量获取（我需要所有修订）。
它再次失败并出现与克隆类似的错误。
我尝试从办公室的光纤宽带 link 进行克隆，它似乎运行良好。
我怀疑 git 存在超时问题。你能帮我解决这个问题吗？
更新：该仓库仅支持 https，不支持 ssh。
谢谢，
斯里尼瓦萨·普拉迪普
您可以使用 git bundle 到 "bundle" 存储库到单个文件中，并通过可以恢复中断下载的 http 服务器获取它。
根据执行克隆的工作站上的处理器内核数量，使用 Git 2.29（2020 年第四季度）时情况可能会有所改善。
很久以前，运行 并行 index-pack 任务时默认只使用 3 个线程：已经向上调整了一点。
参见 commit fbff95b, commit 218389b, commit 4727425 (21 Aug 2020) by Jeff King (peff)。
（由 Junio C Hamano -- gitster -- in commit 53015c9 合并，2020 年 8 月 31 日）
index-pack: adjust default threading cap
Signed-off-by: Jeff King
Commit b8a2486f15 ("index-pack: support multithreaded delta resolving", 2012-05-06, Git v1.7.11-rc0 -- merge) describes an experiment that shows that setting the number of threads for index-pack higher than 3 does not help.
I repeated that experiment using a more modern version of Git and a more modern CPU and got different results.
Here are timings for p5302 against linux.git run on my laptop, a Core i9-9880H with 8 cores plus hyperthreading (so online-cpus returns 16):
5302.3: index-pack 0 threads 256.28(253.41+2.79)
5302.4: index-pack 1 threads 257.03(254.03+2.91)
5302.5: index-pack 2 threads 149.39(268.34+3.06)
5302.6: index-pack 4 threads 94.96(294.10+3.23)
5302.7: index-pack 8 threads 68.12(339.26+3.89)
5302.8: index-pack 16 threads 70.90(655.03+7.21)
5302.9: index-pack default number of threads 116.91(290.05+3.21)
You can see that wall-clock times continue to improve dramatically up to the number of cores, but bumping beyond that (into hyperthreading territory) does not help (and in fact hurts a little).
Here's the same experiment on a machine with dual Xeon 6230's, totaling 40 cores (80 with hyperthreading):
5302.3: index-pack 0 threads 310.04(302.73+6.90)
5302.4: index-pack 1 threads 310.55(302.68+7.40)
5302.5: index-pack 2 threads 178.17(304.89+8.20)
5302.6: index-pack 5 threads 99.53(315.54+9.56)
5302.7: index-pack 10 threads 72.80(327.37+12.79)
5302.8: index-pack 20 threads 60.68(357.74+21.66)
5302.9: index-pack 40 threads 58.07(454.44+67.96)
5302.10: index-pack 80 threads 59.81(720.45+334.52)
5302.11: index-pack default number of threads 134.18(309.32+7.98)
The results are similar; things stop improving at 40 threads.
Curiously, going from 20 to 40 really doesn't help much, either (and increases CPU time considerably).
So that may represent an actual barrier to parallelism, where we lose out due to context-switching and loss of cache locality, but don't reap the wall-clock benefits due to contention of our coarse-grained locks.
So what's a good default value?
It's clear that the current cap of 3 is too low; our default values are 42% and 57% slower than the best times on each machine.
The results on the 40-core machine imply that 20 threads is an actual barrier regardless of the number of cores, so we'll take that as a maximum.
We get the best results on these machines at half of the online-cpus value. That's presumably a result of the hyperthreading. That's common on multi-core Intel processors, but not necessarily elsewhere.
But if we take it as an assumption, we can perform optimally on hyperthreaded machines and still do much better than the status quo on other machines, as long as we never half below the current value of 3.
So that's what this patch does.
我有一个超过 2 GB 的远程仓库（1 个主分支）。我有大约 2-4 MbPS link 的 ADSL 宽带连接。我试图克隆回购协议。 但是我收到以下错误： fatal: 远端意外挂断 致命的：早期的 EOF 致命：索引包失败
我在 Stack Overflow 和其他地方搜索并尝试克隆一个裸仓库。 后来我尝试增量获取（我需要所有修订）。 它再次失败并出现与克隆类似的错误。
我尝试从办公室的光纤宽带 link 进行克隆，它似乎运行良好。 我怀疑 git 存在超时问题。你能帮我解决这个问题吗？
更新：该仓库仅支持 https，不支持 ssh。
谢谢， 斯里尼瓦萨·普拉迪普
您可以使用 git bundle 到 "bundle" 存储库到单个文件中，并通过可以恢复中断下载的 http 服务器获取它。
根据执行克隆的工作站上的处理器内核数量，使用 Git 2.29（2020 年第四季度）时情况可能会有所改善。
很久以前，运行 并行 index-pack 任务时默认只使用 3 个线程：已经向上调整了一点。
参见 commit fbff95b, commit 218389b, commit 4727425 (21 Aug 2020) by Jeff King (peff)。
（由 Junio C Hamano -- gitster -- in commit 53015c9 合并，2020 年 8 月 31 日）
index-pack: adjust default threading capSigned-off-by: Jeff King
Commit b8a2486f15 ("
index-pack: support multithreaded delta resolving", 2012-05-06, Git v1.7.11-rc0 -- merge) describes an experiment that shows that setting the number of threads for index-pack higher than 3 does not help.I repeated that experiment using a more modern version of Git and a more modern CPU and got different results.
Here are timings for p5302 against
linux.gitrun on my laptop, a Core i9-9880H with 8 cores plus hyperthreading (so online-cpus returns 16):5302.3: index-pack 0 threads 256.28(253.41+2.79) 5302.4: index-pack 1 threads 257.03(254.03+2.91) 5302.5: index-pack 2 threads 149.39(268.34+3.06) 5302.6: index-pack 4 threads 94.96(294.10+3.23) 5302.7: index-pack 8 threads 68.12(339.26+3.89) 5302.8: index-pack 16 threads 70.90(655.03+7.21) 5302.9: index-pack default number of threads 116.91(290.05+3.21)You can see that wall-clock times continue to improve dramatically up to the number of cores, but bumping beyond that (into hyperthreading territory) does not help (and in fact hurts a little).
Here's the same experiment on a machine with dual Xeon 6230's, totaling 40 cores (80 with hyperthreading):
5302.3: index-pack 0 threads 310.04(302.73+6.90) 5302.4: index-pack 1 threads 310.55(302.68+7.40) 5302.5: index-pack 2 threads 178.17(304.89+8.20) 5302.6: index-pack 5 threads 99.53(315.54+9.56) 5302.7: index-pack 10 threads 72.80(327.37+12.79) 5302.8: index-pack 20 threads 60.68(357.74+21.66) 5302.9: index-pack 40 threads 58.07(454.44+67.96) 5302.10: index-pack 80 threads 59.81(720.45+334.52) 5302.11: index-pack default number of threads 134.18(309.32+7.98)The results are similar; things stop improving at 40 threads.
Curiously, going from 20 to 40 really doesn't help much, either (and increases CPU time considerably).
So that may represent an actual barrier to parallelism, where we lose out due to context-switching and loss of cache locality, but don't reap the wall-clock benefits due to contention of our coarse-grained locks.So what's a good default value?
It's clear that the current cap of 3 is too low; our default values are 42% and 57% slower than the best times on each machine.
The results on the 40-core machine imply that 20 threads is an actual barrier regardless of the number of cores, so we'll take that as a maximum.
We get the best results on these machines at half of the online-cpus value. That's presumably a result of the hyperthreading. That's common on multi-core Intel processors, but not necessarily elsewhere.
But if we take it as an assumption, we can perform optimally on hyperthreaded machines and still do much better than the status quo on other machines, as long as we never half below the current value of 3.So that's what this patch does.