how to specify error log file and output file in qsub

bash cluster-computing qsub sungridengine
d.putto picture d.putto · Feb 1, 2012 · Viewed 41.8k times · Source

I have a qsub script as 

#####----submit_job.sh---#####
    #!/bin/sh
    #$ -N job1
    #$ -t 1-100
    #$ -cwd
    SEEDFILE=/home/user1/data1
    SEED=$(sed -n -e "$SGE_TASK_ID p" $SEEDFILE)
    /home/user1/run.sh $SEED

The problem is-- it puts all error and output files (job1.eJOBID & job1.oJOBID) to the same directory from where I am running qsub submit_job.sh while I want to save these file (output and error log file in same different place (specified as $SEED_output). I tried to change the line as

/home/user1/run.sh $SEED -o $SEED_output

But it didn't work. Any suggestion ?? How can I specify path and name of default output and error log file ??

Answer

Andy picture Andy · Feb 2, 2012

Typically error and output files are given as pbs directives in the qsub script or as command line options to the qsub script like so:

#! /bin/bash
#PBS -q queue_name
#PBS -A account_name
#PBS -l nodes=12:ppn=12
#PBS -l walltime=18:00:00
#PBS -e /mypath/error.txt
#PBS -o /mypath/output.txt

or as a command line option to qsub like so:

qsub -o /mypath/output.txt -e /mypath/error.txt submit_job.sh

With the first option I don't think you can use a variable as the shell won't look at lines that are commented. Plus I think PBS deals with the commented lines before the shell would. If you know the path when you're invoking qsub, you could try the second option. Alternatively you could try simply redirecting the output and error in the script itself:

/home/user1/run.sh $SEED > ${SEED}/output.txt 2> ${SEED}/error.txt

The third option is probably the easiest. Output and error files might still be created in the run directory, though they'd likely be empty.

Related questions

How to concatenate string variables in Bash

In PHP, strings are concatenated together as follows: $foo = "Hello"; $foo .= " World"; Here, $foo becomes "Hello World". How is this accomplished in Bash?

Read More
How can I check if a directory exists in a Bash shell script?

What command can be used to check if a directory exists or not, within a Bash shell script?

Read More
How do I tell if a regular file does not exist in Bash?

I've used the following script to see if a file exists: #!/bin/bash FILE=$1 if [ -f $FILE ]; then echo "File $FILE exists." else echo "File $FILE does not exist." fi What's the correct syntax to use if I only want …

Read More