Updates and Results Talks and Posters Advice Ideas Important Figures Write-Ups Outreach How-To Funding Opportunities GENETIS
  Place to document instructions for how to do things  ELOG logo
Message ID: 36     Entry time: Mon Mar 4 14:11:07 2019
Author: Jorge Torres 
Subject: Submitting arrays of jobs on OSC 
Project: Analysis 

PBS has the option of having arrays of jobs in case you want to easily submit multiple similar jobs. The only difference in them is the array index, which you can use in your PBS script to run each task with a different set of input arguments, or any other operation that requires a unique index.

You need to add the following lines to your submitter file:

#PBS -t array_min-array_max%increment

where "array_min/array_max" are integers that set lower and upper limit, respectively, of your "job loop" and increment lets you set the number of jobs that you want to submit simultaneously. For example:

#PBS -t 1-100%5

submits an array with 100 jobs in it, but the system will only ever have 5 running at one time.

Here's an example of a script that submits to Pitzer a job array (from 2011-3000 in batches of 40) of a script named "make_fits_noise" that uses one core only. Make sure you use "#PBS -m n", otherwise you'll get tons of emails notyfing you about your jobs.

To delete the whole array, use "qdel JOB_ID[ ]". To delete one single instance, use "qdel JOB_ID[ARRAY_ID]".

More info: https://arc-ts.umich.edu/software/torque/job-arrays/

 

ELOG V3.1.5-fc6679b