[torqueusers] help with unwanted concurrency in a torque script
jbernstein at penguincomputing.com
Mon Dec 7 17:57:26 MST 2009
What you are probably looking for is a feature called job dependencies.
Basically you say that the reporting jobs are Dependant on the population
script, and thus will not run until the population script completes.
Look towards the -W option to qsub:
$ qsub .... -W depend=...
In a nutshell the arguments to depend= are job identifiers. So you can submit
the first populate job, and then every analysis gets submitted to depend on that
Senior Software Engineer
Dan Kortschak wrote:
> I am having problems with unexpected concurrency (at least unexpected
> from my perspective) with a torque script. I am using pbs to manage a
> batch queue on my workstation and at the moment am using it to run a set
> of perl scripts that perform postgresql control.
> The first script populates (eg populate below) the database and then
> subsequent scripts (eg report and extract below) query to perform an
> analysis of the data. The problem is that the report and extract scripts
> appear to start before postgresql has completed the transactions, so the
> reports are erroneous. This does not happen if the torque script is run
> as a csh script.
> Can any one suggest how to avoid this (without hooking in postgresql
> process status communication)? And also where I should look to find out
> more about this?
> ### Job name
> #PBS -N test
> ### Join queuing system output and error files into a single output file
> #PBS -j oe
> #PBS -q sequential
> #PBS -l ncpus=1
> #PBS -l mem=5gb,vmem=2gb
> #PBS -l walltime=70:00:00
> # Run the executables
> populate --infile data
> torqueusers mailing list
> torqueusers at supercluster.org
More information about the torqueusers