[torqueusers] Job array throttling

Gareth.Williams at csiro.au Gareth.Williams at csiro.au
Thu Jul 10 18:18:24 MDT 2008

Hi Gabe,

I suggest that you fix the policy... I can't see a lot of point in
having array jobs only up to a size of ten - and that subject to the
user having no existing queued jobs.

You might look through the archives of the mailing list for info on
setting up per user limits on queued jobs and having routing queues (to
hold as many jobs as you like) that feed into the limited queues.


Gareth Williams
Outreach Manager
Ph +61 3 8601 3804 (CSIRO)
Ph +61 3 9669 8114 (BoM)
calendar: http://tinyurl.com/3mkl5d

> -----Original Message-----
> From: Gabe Turner [mailto:gabe at msi.umn.edu]
> Sent: Friday, 11 July 2008 12:06 AM
> To: torqueusers at supercluster.org
> Subject: [torqueusers] Job array throttling
> We starting to play with job arrays as we have some actual requests
> them from former SGE users which we've recently adopted.  Submitting
> delting them is working fine (in 2.3.0), but there seems to be no way
> I can tell to throttle them.  Basically, I would like to limit the
size of
> any given job array to a static number of jobs.  We have a policy, as
> unwise and politically-motivated as it may be, dictating that no user
> submit more than 10 jobs.  Currently we throttle this using a submit
> filter.
> What our submit filter does is just parse qstat output to determine
> number of jobs a user has submitted (crude, but it works) and it
> an
> error when they try to submit jobs beyond 10.  The problem is that a
> would be able to circumvent this by submitting a job array containing
> than 10 jobs, as it seems the submit filter is only executed once,
> when sumitting a job array.  Fortunately, once this 10+ job array has
> submitted, our filter then prevent further submissions for that user.
> We can work support for detecting '#PBS -t ...' into our submit
> but
> I don't believe that will work if -t is passed on the qsub command
> I've considered the implementation of a qsub wrapper, but frankly it's
> just
> too easy to circumvent.
> Anyone want to brainstorm with me about this?  Any plans by those
> developers working on job arrays to providing throttling policies for
> array size?
> Any help would be greatly appreciated!
> Gabe
> --
> Gabe Turner
gabe at msi.umn.edu
> UNIX System Administrator,
> University of Minnesota
> Supercomputing Institute

More information about the torqueusers mailing list