[Mauiusers] maui limits? looking for experience

Gareth.Williams at csiro.au Gareth.Williams at csiro.au
Thu Sep 29 15:23:52 MDT 2011


> -----Original Message-----
> From: Arnau Bria [mailto:arnaubria at pic.es]
> Sent: Thursday, 29 September 2011 7:56 PM
> To: mauiusers at supercluster.org
> Subject: Re: [Mauiusers] maui limits? looking for experience
> 
> On Wed, 28 Sep 2011 11:15:50 -0400
> Michel Béland wrote:
> 
> > Hi,
> Hi,
> 
> > I would advise defining a limit on idle jobs per user. For example:
> >
> > USERCFG[DEFAULT] MAXIJOB=200
> >
> > or any suitable number for you site.
> 
> This really improves maui behaviour. But limiting idle queue was the
> last thing I wanted to do....

Idle limits are mostly good.  This mostly limits the number of each users jobs that maui will consider in any scheduling cycle so it make the scheduling cycle shorter/faster.  It also limits the priority accumulated by queued jobs and alleviates 'queue stuffing'. I'd recommend idle limits given that maui does not contain a better facility to handle such issues.

> 
> > Alternatively, Torque has a per-queue max_user_queuable setting, but
> > it counts both running and queued jobs. If you use a route queue to
> > route your job to an execution queue, you can define this for the
> > execution queue and jobs will be moved to the execution queue only
> > when the limit is respected.
> 
> If I understand routing queues properly, they send jobs based on job
> required resources. our jobs do not require any special resource, our
> users send jobs based on queue name that show time limits. So, I think
> that routing queues can't help here.

What is being proposed is that you have a routing queue setup with no special resources, just one routing queue per execution queue (but make it as fancy as you like - though simple is good).  Put a limit on the number of (users) jobs in the execution queue(s) (enough to fill the cluster) but allow many jobs in the routing queue(s).  Maui only need consider the execution queue so it's job becomes simpler and it can be faster.

Gareth (who used maui for some time but doesn't now)

> > Both solutions should decrease the load on Maui as it does not need
> > to schedule as many jobs at a time.
> >
> Many thanks for your reply,
> Cheers,
> Arnau



More information about the mauiusers mailing list