Pass command line arguments via sbatch

Jason picture Jason · Dec 30, 2014 · Viewed 30.3k times · Source

Suppose that I have the following simple bash script which I want to submit to a batch server through SLURM:

#!/bin/bash

#SBATCH -o "outFile"$1".txt"
#SBATCH -e "errFile"$1".txt"

hostname

exit 0

In this script, I simply want to write the output of hostname on a textfile whose full name I control via the command-line, like so:

login-2:jobs$ sbatch -D `pwd` exampleJob.sh 1
Submitted batch job 203775

Unfortunately, it seems that my last command-line argument (1) is not parsed through sbatch, since the files created do not have the suffix I'm looking for and the string "$1" is interpreted literally:

login-2:jobs$ ls
errFile$1.txt  exampleJob.sh outFile$1.txt

I've looked around places in SO and elsewhere, but I haven't had any luck. Essentially what I'm looking for is the equivalent of the -v switch of the qsub utility in Torque-enabled clusters.

Edit: As mentioned in the underlying comment thread, I solved my problem the hard way: instead of having one single script that would be submitted several times to the batch server, each with different command line arguments, I created a "master script" that simply echoed and redirected the same content onto different scripts, the content of each being changed by the command line parameter passed. Then I submitted all of those to my batch server through sbatch. However, this does not answer the original question, so I hesitate to add it as an answer to my question or mark this question solved.

Answer

MasterHD picture MasterHD · Jul 24, 2015

I thought I'd offer some insight because I was also looking for the replacement to the -v option in qsub, which for sbatch can be accomplished using the --export option. I found a nice site here that shows a list of conversions from Torque to Slurm, and it made the transition much smoother.

You can specify the environment variable ahead of time in your bash script:

$ var_name='1'
$ sbatch -D `pwd` exampleJob.sh --export=var_name

Or define it directly within the sbatch command just like qsub allowed:

$ sbatch -D `pwd` exampleJob.sh --export=var_name='1'

Whether this works in the # preprocessors of exampleJob.sh is also another question, but I assume that it should give the same functionality found in Torque.