[Mauiusers] Questing regarding Partition, and looking for other alternatives

Gabe Turner gabe at msi.umn.edu
Thu Apr 16 13:57:50 MDT 2009

On Thu, Apr 16, 2009 at 01:55:30PM -0500, Ling C. Ho wrote:
> We have two group of nodes running jobs submitted to two groups of Torque queues. For example, 
> node001-node100 run jobs submitted to queue a, b and c, while node101-node200 run jobs submitted to 
> z, y and z only
> We tried splitting them into two partitions in Maui, using nodecfg for each node. eg. 
> .
> .
> .
> .
> .
> .
> This setup works pretty well, but having to maintain cfg file with the number of nodes make me feel 
> uneasy.
> Jobs for queue a,b,c usually have higher so they are always at the top of the priority table (based 
> on diagnose -p). Without using partitioning, we could never get any job for queue x,y,z started on 
> node101-200 even when node001-100 are all busy. Even when maui is not processing any jobs in queue 
> a,b,c, it doesn't do anything to jobs from x,y,z waiting to start. I have tried using BACKFILL like 
> this and it doesn't work:
> BACKFILLDEPTH           0
> BFCHUNKDURATION         00:00:30
> BFCHUNKSIZE             1
> My questions are,
> - is there another way to assign a node to a partition other than having to enter NOCFG for each and 
> every node in the maui.cfg file? I have 800+ nodes so the cfg file is going to be huge.
> - beside using partition, what other features are appropriate for managing jobs for two non-overlap 
> group of machines like ours?
> - Should like look into using NODESET ?  (Is it even a Maui feature, or just Moab?)

I've always thought of partitions as impenetrable boundries in Maui and
Moab.  I use them as the highest level container, and I never want jobs or
reservations to span them.

To this end, I use NODECFG to assign my nodes to a partition, and then
CLASSCFG to assign my queues to a partition:

    NODECFG[node001]	PARTITION=part1


My users then submit to queue and their job is guaranteed to be in the
right partition.

Now, having job reservations per partition is a bit trickier, but it's
doable (I do it in Moab, so YMMV with Maui).  What I would do is assign a
default QOS for each of your queues

    CLASSCFG[a]		QDEF=part1qos

and add to that QOS a reservation profile that has it's own

    RESERVATIONDEPTH[part1rsv]		1
    RESERVATIONQOSLIST[part1rsv]	part1qos

Then you would also make that QOS the default for your other queues in that

    CLASSCFG[b]		QDEF=part1qos
    CLASSCFG[c]		QDEF=part1qos

I've been using a configuration in Moab similar to this for almost 3 years
and it's been working just fine.  I did my best to map Moab's parameters to
Maui ones using the Maui docs, but you will need to do some


Gabe Turner                                             gabe at msi.umn.edu
HPC Systems Administrator,
University of Minnesota
Supercomputing Institute                          http://www.msi.umn.edu

More information about the mauiusers mailing list