Cannot push to remote git repo over ssh, stuck at "Writing objects: 11%"

Question

I'm having a problem where trying to do a git push hangs up while "Writing objects". This is not the common scenario of Windows and git-daemon that I have seen answered elsewhere. This is Linux to Linux, using ssh. On both the client and server, git version is 1.7.8.6.

I've tried doing the following:

• Create a local empty repo and push to that -- works fine
• Pull from local repo to local repo -- works fine
• Clone the remote repo to the local machine -- works fine
• git fsck and git gc on both server and client repos
• Push to pre-existing remote repo -- hangs writing objects at 11%
• Create empty remote repo and push to that -- hangs writing objects at 2%

There's an env variable I can set to make git produce debug messages. Nothing useful there.

I've talked to people at length in IRC about this, and I've tried googling it. We're getting nowhere. Can anyone here help me figure out what's going wrong?

Thanks!

UPDATE: Based on someone's suggestion, I did an strace -F, and it gets stuck on a select() in a child process. I'd provide the whole trace, but stackexchange doesn't have a way to upload. Anyhow, here's the relevant section:

[pid 14414] getsockopt(3, SOL_SOCKET, SO_RCVBUF, [87872], [4]) = 0^M
[pid 14414] select(7, [3 4], [3], NULL, NULL) = 2 (in [4], out [3])^M
[pid 14414] read(4,  <unfinished ...>^M
[pid 14433] <... write resumed> )       = 8192^M
[pid 14433] write(1, "\262\36\237\265\234\n\332W\26\271.\250\271W\34\250N\34*\341\247R\21\177\\Dq\23\27\211\270\25"..., 8192) = 8192^M
[pid 14433] write(1, "Y\343}z\367\31y\321)\254\236na\274\347*\253\247\264d\25\213\364\243\331\242\236\3222U3\246"..., 8192 <unfinished ...>^M
[pid 14414] <... read resumed> "t\277\177\276\361\377\327U\361Re\f\347\300\27\3\255}E\363\2277\312p\356F\335U\345\336\250\273"..., 16384) = 16384^M
[pid 14414] write(3, "\210y\r\202\303`\222m\216\177x \354\216Y\322n\232l\273\237\214\352\32\r\261r\311\347\356\236\26"..., 16432) = 16432^M
[pid 14414] getsockopt(3, SOL_SOCKET, SO_RCVBUF, [87872], [4]) = 0^M
[pid 14414] select(7, [3], [3], NULL, NULL) = 1 (out [3])^M
[pid 14414] write(3, "\v\267\355\33\257\332\330\326\233\260;\223\212z\333\213\21'1\20\7\366\"f\253\373\0059\24\206C4"..., 5504) = 5504^M
[pid 14414] getsockopt(3, SOL_SOCKET, SO_RCVBUF, [87872], [4]) = 0^M
[pid 14414] select(7, [3], [], NULL, NULL <unfinished ...>^M
[pid 14433] <... write resumed> )       = ? ERESTARTSYS (To be restarted if SA_RESTART is set)^M
[pid 14433] --- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=1, ptr=0x1}} ---^M
[pid 14433] rt_sigreturn()              = 1^M
[pid 14433] write(1, "Y\343}z\367\31y\321)\254\236na\274\347*\253\247\264d\25\213\364\243\331\242\236\3222U3\246"..., 8192) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)^M
[pid 14433] --- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=1, ptr=0x1}} ---^M
[pid 14433] rt_sigreturn()              = 1^M
[pid 14433] write(1, "Y\343}z\367\31y\321)\254\236na\274\347*\253\247\264d\25\213\364\243\331\242\236\3222U3\246"..., 8192) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)^M
[pid 14433] --- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=1, ptr=0x1}} ---^M
[pid 14433] rt_sigreturn()              = 1^M
[pid 14433] write(1, "Y\343}z\367\31y\321)\254\236na\274\347*\253\247\264d\25\213\364\243\331\242\236\3222U3\246"..., 8192) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)^M
[pid 14433] --- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=1, ptr=0x1}} ---^M
[pid 14433] rt_sigreturn()              = 1^M

Answer 1

The problem was a bug in ssh. A recent version broke something, and it stalls out during large transfers. There is a work-around.

The fix is to set "TcpRcvBufPoll no" in sshd_config.

http://forums.gentoo.org/viewtopic-t-925044-start-0-postdays-0-postorder-asc-highlight-.html

http://forums.gentoo.org/viewtopic-p-7036250.html