Process output from apache-commons exec

Question

I am at my wits end here. I'm sure this is something simple and I most likely have huge holes in my understanding of java and streams. I think there are so many classes that I'm a bit overwhelmed with trying to poke through the API to figure out when and how I want to use the multitude of input/output streams.

I just learned about the existence of the apache commons library (self teaching java fail), and am currently trying to convert some of my Runtime.getRuntime().exec to use the commons - exec. Already it's fixed some of the once every 6 months this problem crops up then goes away style problems with exec.

The code executes a perl script, and displays the stdout from the script in the GUI as it is running.

The calling code is inside of a swingworker.

I'm getting lost how to use the pumpStreamHandler... anyway here is the old code:

String pl_cmd = "perl script.pl"
Process p_pl = Runtime.getRuntime().exec( pl_cmd );

BufferedReader br_pl = new BufferedReader( new InputStreamReader( p_pl.getInputStream() ) );

stdout = br_pl.readLine();
while ( stdout != null )
{
    output.displayln( stdout );
    stdout = br_pl.readLine();
}

I guess this is what I get for copy pasting code I don't fully understand a long time ago. The above I assume is executing the process, then grabs the outputstream (via "getInputStream"?), places it into a buffered reader, then will just loop there until the buffer is empty.

What I don't get is why there is no need for a 'waitfor' style command here? Isn't it possible that there will be some time in which the buffer will be empty, exit the loop, and continue on while the process is still going? When I run it, this doesn't seem to be the case.

In any event, I'm trying to get the same behavior using commons exec, basically again going from google found code:

DefaultExecuteResultHandler rh = new DefaultExecuteResultHandler();
ExecuteWatchdog wd  = new ExecuteWatchdog( ExecuteWatchdog.INFINITE_TIMEOUT );
Executor exec = new DefaultExecutor();

ByteArrayOutputStream out = new ByteArrayOutputStream();
PumpStreamHandler psh = new PumpStreamHandler( out );

exec.setStreamHandler( psh );
exec.setWatchdog( wd );

exec.execute(cmd, rh );
rh.waitFor();

I'm trying to figure out what pumpstreamhandler is doing. I assume that this will take the output from the exec object, and fill the OutputStream I provide it with the bytes from the perl script's stdout/err?

If so how would you get the above behavior to have it stream the output line by line? In examples people show you call the out.toString() at the end, and I assume this would just give me a dump of all the output from the script once it is done running? How would you do it such that it would show the output as it is running line by line?

------------Future Edit ---------------------

Found this via google and works nice as well:

public static void main(String a[]) throws Exception
{
    ByteArrayOutputStream stdout = new ByteArrayOutputStream();
    PumpStreamHandler psh = new PumpStreamHandler(stdout);
    CommandLine cl = CommandLine.parse("ls -al");
    DefaultExecutor exec = new DefaultExecutor();
    exec.setStreamHandler(psh);
    exec.execute(cl);
    System.out.println(stdout.toString());
}

Answer 1

Don't pass a ByteArrayOutputStream to the PumpStreamHandler, use an implementation of the abstract class org.apache.commons.exec.LogOutputStream. From the javadoc:

The implementation parses the incoming data to construct a line and passes the complete line to an user-defined implementation.

Thus the LogOutputStram is preprocessing the output to give you the control of handling individual lines instead of the raw bytes. Something like this:

import java.util.LinkedList;
import java.util.List;
import org.apache.commons.exec.LogOutputStream;

public class CollectingLogOutputStream extends LogOutputStream {
    private final List<String> lines = new LinkedList<String>();
    @Override protected void processLine(String line, int level) {
        lines.add(line);
    }   
    public List<String> getLines() {
        return lines;
    }
}

Then after the blocking call to exec.execute your getLines() will have the standard out and standard error you are looking for. The ExecutionResultHandler is optional from the perspective of just executing the process, and collecting all the stdOut/stdErr into a list of lines.