git stderr output can't pipe

Delan Azabani picture Delan Azabani · Oct 31, 2010 · Viewed 9.3k times · Source

I'm writing a graphical URI handler for git:// links with bash and zenity, and I'm using a zenity 'text-info' dialog to show git's clone output while it's running, using FIFO piping. The script is about 90 lines long, so I won't bother posting it here, but here's the most important lines:

git clone "$1" "$target" 2>&1 | cat >> /tmp/githandler-fifo &
cat /tmp/githandler-fifo | zenity --text-info --text='Cloning git repository' &

I'm using FIFO instead of a direct pipe to allow them to run asynchronously and allow for killing git if the zenity window is closed.

Problem is, the only line that appears from git's output is the first:

Initialized empty Git repository in /home/delan/a/.git/

The other lines with counting objects, etc. don't show or are shown on the terminal.

Current reason

The current consensus as to why this isn't working seems to be that cat is non-blocking and quits after the first line, only passing that to zenity and not the rest. My aim is to force blocking for the read, and have zenity's text info dialog show all output progressively.

git outputs progress messages (anything other than the "Initialized" message) on stderr, but the moment I try to pipe stderr to a file or to merge with stdout, the messages disappear.

Fix attempt 1

I've tried to write two blocking versions of cat's functions in C, bread and bwrite, like this:

#include <stdio.h>
main(int argc, char **argv) {
    int c;
    for (;;) {
        freopen(argv[1], "r", stdin);
        while ((c = getchar()) != EOF)
            putchar(c);
    }
}

#include <stdio.h>
main(int argc, char **argv) {
    int c;
    for (;;) {
        freopen(argv[1], "w", stdout);
        while ((c = getchar()) != EOF)
            putchar(c), fputs("writing", stderr);
    }
}

They work nicely because they block and don't quit on EOF, but it hasn't quite solved the problem yet. At the moment, using one, the other, or both, works in theory, but in practice, zenity shows nothing at all now.

Fix attempt 2

@mvds suggested that using a regular file, in combination with tail -f rather than cat, may do this. Surprised at such a simple solution (thanks!) I tried it but unfortunately, only the first line showed up in zenity and nothing else.

Fix attempt 3

After doing some strace'ing and inspecting git's source code, I realise that git outputs all its progress information (anything past the "Initialized" message) on stderr, and the fact that this is the first line and my assumption that it's because of cat quitting early on EOF was a coincidence/misguided assumption (git doesn't EOF until the program ends).

The situation seemed to become a lot simpler, as I shouldn't have to change anything from the original code (at the start of the question) and it should work. Mysteriously, however, the stderr output 'vanishes' when redirected - and this is only something that happens in git.

Test case? Try this, and see if you see anything in the file (you won't):

git clone git://anongit.freedesktop.org/xorg/proto/dri2proto 2> hurr

This goes against everything I know about stderr and redirection; I've even written a little C program that outputs on stderr and stdout to prove to myself that redirection just doesn't work for git.

Fix attempt 4

In line with Jakub Narębski's answer, as well as replies to emails I sent to the git mailing list, --progress is the option I need. Note that this option only works after the command, and not before clone.

Success!

Thank you very much for all your help. This is the fixed line:

git clone "$1" "$target" --progress > /tmp/githandler-fifo 2>&1 &

Answer

Jakub Narębski picture Jakub Narębski · Oct 31, 2010

I think that at least some of progress reports gets silenced when output is not a terminal (tty). I'm not sure if it applies to your case, but try to pass --progress option to 'git clone' (i.e. use git clone --progress <repository>).

Though I don't know if it is what you wanted to have.