git clone gets hung and gets timedout
I use Bitbucket(git) server and currently the git clone is randomly getting hung. The same repo sometimes gets cloned and sometimes get hung unexpectedly and then gets timed out. The server memory, CPU utilization, disk space is fine. Please advise what could be other areas that needs attention to find the root cause.This is happening on 5 different Jenkins slave so it is not restricted to a single server.
Following is the error log:
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "/usr/bin/git -c core.askpass=true fetch --tags --progress http://bitbucket.net/project.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: remote: Counting objects: 2506, done.
stdout:
stderr: remote: Counting objects: 2506, done.
remote: Compressing objects: 0% (1/2405)
remote: Compressing objects: 1% (25/2405)
remote: Compressing objects: 2% (49/2405)
remote: Compressing objects: 3% (73/2405)
remote: Compressing objects: 4% (97/2405)
remote: Compressing objects: 5% (121/2405)
remote: Compressing objects: 6% (145/2405)
remote: Compressing objects: 7% (169/2405)
remote: Compressing objects: 8% (193/2405)
remote: Compressing objects: 9% (217/2405)
remote: Compressing objects: 10% (241/2405)
remote: Compressing objects: 11% (265/2405)
remote: Compressing objects: 12% (289/2405)
remote: Compressing objects: 13% (313/2405)
remote: Compressing objects: 14% (337/2405)
remote: Compressing objects: 15% (361/2405)
remote: Compressing objects: 16% (385/2405)
remote: Compressing objects: 17% (409/2405)
remote: Compressing objects: 18% (433/2405)
remote: Compressing objects: 19% (457/2405)
remote: Compressing objects: 20% (481/2405)
remote: Compressing objects: 21% (506/2405)
remote: Compressing objects: 22% (530/2405)
remote: Compressing objects: 23% (554/2405)
remote: Compressing objects: 24% (578/2405)
remote: Compressing objects: 25% (602/2405)
remote: Compressing objects: 26% (626/2405)
remote: Compressing objects: 27% (650/2405)
remote: Compressing objects: 28% (674/2405)
remote: Compressing objects: 29% (698/2405)
remote: Compressing objects: 30% (722/2405)
remote: Compressing objects: 31% (746/2405)
remote: Compressing objects: 32% (770/2405)
remote: Compressing objects: 33% (794/2405)
remote: Compressing objects: 34% (818/2405)
remote: Compressing objects: 35% (842/2405)
remote: Compressing objects: 36% (866/2405)
remote: Compressing objects: 37% (890/2405)
remote: Compressing objects: 38% (914/2405)
remote: Compressing objects: 39% (938/2405)
remote: Compressing objects: 40% (962/2405)
remote: Compressing objects: 41% (987/2405)
remote: Compressing objects: 42% (1011/2405)
remote: Compressing objects: 43% (1035/2405)
remote: Compressing objects: 44% (1059/2405)
remote: Compressing objects: 45% (1083/2405)
remote: Compressing objects: 46% (1107/2405)
remote: Compressing objects: 47% (1131/2405)
remote: Compressing objects: 48% (1155/2405)
remote: Compressing objects: 49% (1179/2405)
remote: Compressing objects: 50% (1203/2405)
remote: Compressing objects: 51% (1227/2405)
remote: Compressing objects: 52% (1251/2405)
remote: Compressing objects: 53% (1275/2405)
remote: Compressing objects: 54% (1299/2405)
remote: Compressing objects: 55% (1323/2405)
remote: Compressing objects: 56% (1347/2405)
remote: Compressing objects: 57% (1371/2405)
remote: Compressing objects: 58% (1395/2405)
remote: Compressing objects: 59% (1419/2405)
remote: Compressing objects: 60% (1443/2405)
remote: Compressing objects: 61% (1468/2405)
remote: Compressing objects: 62% (1492/2405)
remote: Compressing objects: 63% (1516/2405)
remote: Compressing objects: 64% (1540/2405)
remote: Compressing objects: 65% (1564/2405)
remote: Compressing objects: 66% (1588/2405)
remote: Compressing objects: 67% (1612/2405)
remote: Compressing objects: 68% (1636/2405)
remote: Compressing objects: 69% (1660/2405)
remote: Compressing objects: 70% (1684/2405)
remote: Compressing objects: 71% (1708/2405)
remote: Compressing objects: 72% (1732/2405)
remote: Compressing objects: 73% (1756/2405)
remote: Compressing objects: 74% (1780/2405)
remote: Compressing objects: 75% (1804/2405)
remote: Compressing objects: 76% (1828/2405)
remote: Compressing objects: 77% (1852/2405)
remote: Compressing objects: 78% (1876/2405)
remote: Compressing objects: 79% (1900/2405)
remote: Compressing objects: 80% (1924/2405)
remote: Compressing objects: 81% (1949/2405)
remote: Compressing objects: 82% (1973/2405)
remote: Compressing objects: 83% (1997/2405)
remote: Compressing objects: 84% (2021/2405)
remote: Compressing objects: 85% (2045/2405)
remote: Compressing objects: 86% (2069/2405)
remote: Compressing objects: 87% (2093/2405)
remote: Compressing objects: 88% (2117/2405)
remote: Compressing objects: 89% (2141/2405)
remote: Compressing objects: 90% (2165/2405)
remote: Compressing objects: 91% (2189/2405)
remote: Compressing objects: 92% (2213/2405)
remote: Compressing objects: 93% (2237/2405)
remote: Compressing objects: 94% (2261/2405)
remote: Compressing objects: 94% (2278/2405)
remote: Compressing objects: 95% (2285/2405)
remote: Compressing objects: 96% (2309/2405)
remote: Compressing objects: 97% (2333/2405)
remote: Compressing objects: 98% (2357/2405)
remote: Compressing objects: 99% (2381/2405)
remote: Compressing objects: 100% (2405/2405)
remote: Compressing objects: 100% (2405/2405), done.
Receiving objects: 0% (1/2506)
Receiving objects: 1% (26/2506)
Receiving objects: 2% (51/2506)
Receiving objects: 3% (76/2506)
Receiving objects: 4% (101/2506)
Receiving objects: 5% (126/2506)
Receiving objects: 6% (151/2506)
Receiving objects: 7% (176/2506)
Receiving objects: 8% (201/2506)
Receiving objects: 9% (226/2506)
Receiving objects: 10% (251/2506)
Receiving objects: 11% (276/2506)
Receiving objects: 12% (301/2506)
Receiving objects: 13% (326/2506)
Receiving objects: 14% (351/2506)
Receiving objects: 15% (376/2506)
Receiving objects: 16% (401/2506)
Receiving objects: 17% (427/2506)
Receiving objects: 18% (452/2506)
Receiving objects: 19% (477/2506)
Receiving objects: 20% (502/2506)
Receiving objects: 21% (527/2506)
Receiving objects: 22% (552/2506)
Receiving objects: 23% (577/2506)
Receiving objects: 24% (602/2506)
Receiving objects: 25% (627/2506)
Receiving objects: 26% (652/2506)
Receiving objects: 27% (677/2506)
Receiving objects: 28% (702/2506)
Receiving objects: 29% (727/2506)
Receiving objects: 30% (752/2506)
Receiving objects: 31% (777/2506)
Receiving objects: 32% (802/2506)
Receiving objects: 33% (827/2506)
Receiving objects: 34% (853/2506)
Receiving objects: 35% (878/2506)
Receiving objects: 36% (903/2506)
Receiving objects: 37% (928/2506)
Receiving objects: 38% (953/2506)
Receiving objects: 39% (978/2506)
Receiving objects: 40% (1003/2506)
Receiving objects: 41% (1028/2506)
Receiving objects: 42% (1053/2506)
Receiving objects: 43% (1078/2506)
Receiving objects: 44% (1103/2506)
Receiving objects: 45% (1128/2506)
Receiving objects: 46% (1153/2506)
Receiving objects: 47% (1178/2506)
Receiving objects: 48% (1203/2506)
Receiving objects: 49% (1228/2506)
Receiving objects: 50% (1253/2506)
Receiving objects: 51% (1279/2506)
Receiving objects: 52% (1304/2506)
Receiving objects: 53% (1329/2506)
Receiving objects: 54% (1354/2506)
Receiving objects: 55% (1379/2506)
Receiving objects: 56% (1404/2506)
Receiving objects: 57% (1429/2506)
Receiving objects: 58% (1454/2506)
Receiving objects: 59% (1479/2506)
Receiving objects: 60% (1504/2506)
Receiving objects: 61% (1529/2506)
Receiving objects: 62% (1554/2506)
Receiving objects: 63% (1579/2506)
Receiving objects: 64% (1604/2506)
Receiving objects: 65% (1629/2506)
Receiving objects: 66% (1654/2506)
Receiving objects: 67% (1680/2506)
Receiving objects: 68% (1705/2506)
Receiving objects: 69% (1730/2506)
Receiving objects: 70% (1755/2506)
Receiving objects: 71% (1780/2506)
Receiving objects: 72% (1805/2506)
Receiving objects: 73% (1830/2506)
Receiving objects: 74% (1855/2506)
Receiving objects: 75% (1880/2506), 188.00 KiB | 367.00 KiB/s
Receiving objects: 76% (1905/2506), 188.00 KiB | 367.00 KiB/s
Receiving objects: 77% (1930/2506), 188.00 KiB | 367.00 KiB/s
Receiving objects: 78% (1955/2506), 188.00 KiB | 367.00 KiB/s
Receiving objects: 78% (1958/2506), 412.00 KiB | 406.00 KiB/s
Receiving objects: 79% (1980/2506), 412.00 KiB | 406.00 KiB/s
Receiving objects: 80% (2005/2506), 652.00 KiB | 429.00 KiB/s
Receiving objects: 81% (2030/2506), 652.00 KiB | 429.00 KiB/s
Receiving objects: 82% (2055/2506), 652.00 KiB | 429.00 KiB/s
Receiving objects: 83% (2080/2506), 652.00 KiB | 429.00 KiB/s
Receiving objects: 84% (2106/2506), 652.00 KiB | 429.00 KiB/s
Receiving objects: 85% (2131/2506), 652.00 KiB | 429.00 KiB/s
Receiving objects: 86% (2156/2506), 652.00 KiB | 429.00 KiB/s
Receiving objects: 86% (2170/2506), 964.00 KiB | 468.00 KiB/s
Receiving objects: 87% (2181/2506), 964.00 KiB | 468.00 KiB/s
Receiving objects: 88% (2206/2506), 964.00 KiB | 468.00 KiB/s
Receiving objects: 89% (2231/2506), 964.00 KiB | 468.00 KiB/s
Receiving objects: 90% (2256/2506), 964.00 KiB | 468.00 KiB/s
Receiving objects: 90% (2266/2506), 1.32 MiB | 416.00 KiB/s
Receiving objects: 91% (2281/2506), 1.32 MiB | 416.00 KiB/s
Receiving objects: 92% (2306/2506), 1.32 MiB | 416.00 KiB/s
Receiving objects: 92% (2312/2506), 1.54 MiB | 411.00 KiB/s
Receiving objects: 92% (2319/2506), 1.71 MiB | 347.00 KiB/s
Receiving objects: 93% (2331/2506), 1.77 MiB | 306.00 KiB/s
Receiving objects: 93% (2333/2506), 1.77 MiB | 306.00 KiB/s
Receiving objects: 93% (2341/2506), 1.82 MiB | 222.00 KiB/s
Receiving objects: 93% (2344/2506), 1.88 MiB | 112.00 KiB/s
Receiving objects: 93% (2345/2506), 1.90 MiB | 97.00 KiB/s
Receiving objects: 93% (2348/2506), 1.93 MiB | 65.00 KiB/s
Receiving objects: 94% (2356/2506), 1.93 MiB | 65.00 KiB/s
Receiving objects: 94% (2371/2506), 1.98 MiB | 46.00 KiB/s
Receiving objects: 94% (2371/2506), 2.01 MiB | 34.00 KiB/s
Receiving objects: 94% (2372/2506), 2.01 MiB | 34.00 KiB/s
Receiving objects: 94% (2372/2506), 2.04 MiB | 29.00 KiB/s
Receiving objects: 94% (2373/2506), 2.05 MiB | 28.00 KiB/s
Receiving objects: 94% (2375/2506), 2.07 MiB | 22.00 KiB/s
Receiving objects: 94% (2376/2506), 2.09 MiB | 21.00 KiB/s
Receiving objects: 95% (2381/2506), 2.11 MiB | 19.00 KiB/s
error: index-pack died of signal 15
fatal: index-pack failed
Answer
I suggest you set the git clone timeout value upto a logger value, the default is 10 minutes. It would cause error if the git repo is big enough or the networking is not good.
Here is how to change the default git clone timeout value:
Source Code Management --> Git --> Add--> Additional Behaviours --> Advanced clone behaviors --> Timeout (in minutes) for clone and fetch operations
You can set it to 60 mins for example.