[Mauiusers] Large queues cause Maui to idle

Steve Young chemadm at hamilton.edu
Wed Dec 10 13:57:02 MST 2008


I've used a routing queue to solve this problem. The queue that the  
user is running on can only utilize 32 cpu's. The thousands of jobs  
are 1 cpu each. So I have this for a routing queue:

create queue physics
set queue physics queue_type = Route
set queue physics acl_group_enable = True
set queue physics route_destinations += herc
set queue physics enabled = True
set queue physics started = True

So jobs that go into here are moved to the herc execution queue. This  
queue has the following setting:

set queue herc max_queuable = 36

This way only 36 jobs at time can be queue'd from the routing queue.  
This way maui doesn't even have to worry about considering each of all  
the thousand's of jobs each iteration. It only has to worry about  
scheduling the jobs for the resources it has to run on.

I also use MAXIJOB in maui:

CLASSCFG[herc]		QLIST=md QDEF=md MAXIJOB=4

This way even if a user had lots of jobs in the queue only their top 4  
idle jobs will get considered for scheduling. This way others will be  
able to get their jobs to run without having to wait for maui to  
process thousands of jobs that can't run yet anyhow.

I hope this helps.

-Steve




On Dec 10, 2008, at 3:38 PM, Nicholas Geraedts wrote:

> Thanks Troy and Halvor. You were both correct about the MMAX_JOB  
> definition in the .h file. I've increased it and asked the user to  
> try to break the system again. I'll let you know how things go.
>
> Cheers,
> -Nick
>
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/mauiusers



More information about the mauiusers mailing list