How can I read the error output of external commands in Perl?

Question

As part of a larger Perl program, I am checking the outputs of diff commands of input files in a folder against reference files, where a blank output (a match) is a passing result, and any output from diff is a fail result.

The issue is, if the target folder is short on the number of expected files, the exception diff throws doesn't come as output, creating false passes.

Output Example:

diff: /testfolder/Test-02/test-output.2: No such file or directory

Test-01: PASS

Test-02: PASS

The code goes as such:

$command = "(diff call on 2 files)";
my @output = `$command`;
print "Test-02: ";
$toPrint = "PASS";
foreach my $x (@output) {
    if ($x =~ /./) {
        $toPrint = "FAIL";
    }
}

This is a quick hackery job to fail if there is any output from the diff call. Is there a way to check for exceptions thrown by the command called in the backticks?

Answer 1

There's the answer in perlfaq8: How can I capture STDERR from an external command?

---

There are three basic ways of running external commands:

system $cmd;        # using system()
$output = `$cmd`;       # using backticks (``)
open (PIPE, "cmd |");   # using open()

With system(), both STDOUT and STDERR will go the same place as the script's STDOUT and STDERR, unless the system() command redirects them. Backticks and open() read only the STDOUT of your command.

You can also use the open3() function from IPC::Open3. Benjamin Goldberg provides some sample code:

To capture a program's STDOUT, but discard its STDERR:

use IPC::Open3;
use File::Spec;
use Symbol qw(gensym);
open(NULL, ">", File::Spec->devnull);
my $pid = open3(gensym, \*PH, ">&NULL", "cmd");
while( <PH> ) { }
waitpid($pid, 0);

To capture a program's STDERR, but discard its STDOUT:

use IPC::Open3;
use File::Spec;
use Symbol qw(gensym);
open(NULL, ">", File::Spec->devnull);
my $pid = open3(gensym, ">&NULL", \*PH, "cmd");
while( <PH> ) { }
waitpid($pid, 0);

To capture a program's STDERR, and let its STDOUT go to our own STDERR:

use IPC::Open3;
use Symbol qw(gensym);
my $pid = open3(gensym, ">&STDERR", \*PH, "cmd");
while( <PH> ) { }
waitpid($pid, 0);

To read both a command's STDOUT and its STDERR separately, you can redirect them to temp files, let the command run, then read the temp files:

use IPC::Open3;
use Symbol qw(gensym);
use IO::File;
local *CATCHOUT = IO::File->new_tmpfile;
local *CATCHERR = IO::File->new_tmpfile;
my $pid = open3(gensym, ">&CATCHOUT", ">&CATCHERR", "cmd");
waitpid($pid, 0);
seek $_, 0, 0 for \*CATCHOUT, \*CATCHERR;
while( <CATCHOUT> ) {}
while( <CATCHERR> ) {}

But there's no real need for both to be tempfiles... the following should work just as well, without deadlocking:

use IPC::Open3;
use Symbol qw(gensym);
use IO::File;
local *CATCHERR = IO::File->new_tmpfile;
my $pid = open3(gensym, \*CATCHOUT, ">&CATCHERR", "cmd");
while( <CATCHOUT> ) {}
waitpid($pid, 0);
seek CATCHERR, 0, 0;
while( <CATCHERR> ) {}

And it'll be faster, too, since we can begin processing the program's stdout immediately, rather than waiting for the program to finish.

With any of these, you can change file descriptors before the call:

open(STDOUT, ">logfile");
system("ls");

or you can use Bourne shell file-descriptor redirection:

$output = `$cmd 2>some_file`;
open (PIPE, "cmd 2>some_file |");

You can also use file-descriptor redirection to make STDERR a duplicate of STDOUT:

$output = `$cmd 2>&1`;
open (PIPE, "cmd 2>&1 |");

Note that you cannot simply open STDERR to be a dup of STDOUT in your Perl program and avoid calling the shell to do the redirection. This doesn't work:

open(STDERR, ">&STDOUT");
$alloutput = `cmd args`;  # stderr still escapes

This fails because the open() makes STDERR go to where STDOUT was going at the time of the open(). The backticks then make STDOUT go to a string, but don't change STDERR (which still goes to the old STDOUT).

Note that you must use Bourne shell (sh(1)) redirection syntax in backticks, not csh(1)! Details on why Perl's system() and backtick and pipe opens all use the Bourne shell are in the versus/csh.whynot article in the "Far More Than You Ever Wanted To Know" collection in http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz . To capture a command's STDERR and STDOUT together:

$output = `cmd 2>&1`;                       # either with backticks
$pid = open(PH, "cmd 2>&1 |");              # or with an open pipe
while (<PH>) { }                            #    plus a read

To capture a command's STDOUT but discard its STDERR:

$output = `cmd 2>/dev/null`;                # either with backticks
$pid = open(PH, "cmd 2>/dev/null |");       # or with an open pipe
while (<PH>) { }                            #    plus a read

To capture a command's STDERR but discard its STDOUT:

$output = `cmd 2>&1 1>/dev/null`;           # either with backticks
$pid = open(PH, "cmd 2>&1 1>/dev/null |");  # or with an open pipe
while (<PH>) { }                            #    plus a read

To exchange a command's STDOUT and STDERR in order to capture the STDERR but leave its STDOUT to come out our old STDERR:

$output = `cmd 3>&1 1>&2 2>&3 3>&-`;        # either with backticks
$pid = open(PH, "cmd 3>&1 1>&2 2>&3 3>&-|");# or with an open pipe
while (<PH>) { }                            #    plus a read

To read both a command's STDOUT and its STDERR separately, it's easiest to redirect them separately to files, and then read from those files when the program is done:

system("program args 1>program.stdout 2>program.stderr");

Ordering is important in all these examples. That's because the shell processes file descriptor redirections in strictly left to right order.

system("prog args 1>tmpfile 2>&1");
system("prog args 2>&1 1>tmpfile");

The first command sends both standard out and standard error to the temporary file. The second command sends only the old standard output there, and the old standard error shows up on the old standard out.