[Mauiusers] Large queues cause Maui to idle
ngeraedts at gmail.com
Wed Dec 10 15:36:58 MST 2008
Thank you Steve! Your solution is exactly what we're looking for and seems
to work quite well.
On Wed, Dec 10, 2008 at 12:57 PM, Steve Young <chemadm at hamilton.edu> wrote:
> I've used a routing queue to solve this problem. The queue that the user is
> running on can only utilize 32 cpu's. The thousands of jobs are 1 cpu each.
> So I have this for a routing queue:
> create queue physics
> set queue physics queue_type = Route
> set queue physics acl_group_enable = True
> set queue physics route_destinations += herc
> set queue physics enabled = True
> set queue physics started = True
> So jobs that go into here are moved to the herc execution queue. This queue
> has the following setting:
> set queue herc max_queuable = 36
> This way only 36 jobs at time can be queue'd from the routing queue. This
> way maui doesn't even have to worry about considering each of all the
> thousand's of jobs each iteration. It only has to worry about scheduling the
> jobs for the resources it has to run on.
> I also use MAXIJOB in maui:
> CLASSCFG[herc] QLIST=md QDEF=md MAXIJOB=4
> This way even if a user had lots of jobs in the queue only their top 4 idle
> jobs will get considered for scheduling. This way others will be able to get
> their jobs to run without having to wait for maui to process thousands of
> jobs that can't run yet anyhow.
> I hope this helps.
> On Dec 10, 2008, at 3:38 PM, Nicholas Geraedts wrote:
> Thanks Troy and Halvor. You were both correct about the MMAX_JOB
>> definition in the .h file. I've increased it and asked the user to try to
>> break the system again. I'll let you know how things go.
>> mauiusers mailing list
>> mauiusers at supercluster.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the mauiusers