[torqueusers] help with unwanted concurrency in a torque script

[Joshua Bernstein]

Hey Dan,

	What you are probably looking for is a feature called job dependencies. 
Basically you say that the reporting jobs are Dependant on the population 
script, and thus will not run until the population script completes.

http://www.clusterresources.com/torquedocs21/commands/qsub.shtml

Look towards the -W option to qsub:

$ qsub .... -W depend=...

In a nutshell the arguments to depend= are job identifiers. So you can submit 
the first populate job, and then every analysis gets submitted to depend on that 
job.

-Joshua Bernstein
Senior Software Engineer
Penguin Computing



Dan Kortschak wrote:
> Hi,
> 
> I am having problems with unexpected concurrency (at least unexpected
> from my perspective) with a torque script. I am using pbs to manage a
> batch queue on my workstation and at the moment am using it to run a set
> of perl scripts that perform postgresql control.
> 
> The first script populates (eg populate below) the database and then
> subsequent scripts (eg report and extract below) query to perform an
> analysis of the data. The problem is that the report and extract scripts
> appear to start before postgresql has completed the transactions, so the
> reports are erroneous. This does not happen if the torque script is run
> as a csh script.
> 
> Can any one suggest how to avoid this (without hooking in postgresql
> process status communication)? And also where I should look to find out
> more about this?
> 
> thanks
> 
> 
> #!/bin/csh
> 
> ### Job name
> #PBS -N test
> 
> ### Join queuing system output and error files into a single output file
> #PBS -j oe
> 
> #PBS -q sequential
> 
> #PBS -l ncpus=1
> #PBS -l mem=5gb,vmem=2gb
> #PBS -l walltime=70:00:00
> 
> # Run the executables
> populate --infile data
> report
> extract
> 
> 
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers