[Mauiusers] per-queue group priority setting
Peter Ruprecht
ruprech at jilau1.Colorado.EDU
Wed Jul 23 09:29:53 MDT 2008
On Thu, 3 Jul 2008, Corey Ferrier wrote:
> On Wed, Jul 02, 2008 at 01:21:47PM -0600, Peter Ruprecht wrote:
> >
> >Hi,
> >
> >I'm running maui 3.2 as a scheduler for torque 2.1.8. Suppose I have
> >two queues, called "xeon" and "opteron". Also, there are two groups
> >of users, called "smith" and "jones".
> >
> >I'd like both groups to be able to submit jobs to either queue, but for
> >the smith group to have higher default priority on xeon, while jones
> >has higher priority on opteron.
> >
> >Is there some combination of QOSCFG, GROUPCFG, and PRIORITY that will
> >enable this? For example if I use just
> >
> >GROUPCFG[smith] PRIORITY=20000
> >
> >there's no indication of which queue it applies to.
> >
> >Thanks for any suggestions,
> >
>
> Would this work?
>
> CLASSCFG[xeon] QLIST=xeonprio:normal
> CLASSCFG[opteron] QLIST=opteronprio:normal
>
> QOSCFG[xeonprio] PRIORITY=20000
> QOSCFG[opteronprio] PRIORITY=20000
> QOSCFG[normal] PRIORITY=1
>
> GROUPCFG[smith] QDEF=xeonprio QLIST=xeonprio:normal
> GROUPCFG[jones] QDEF=opteronprio QLIST=opteronprio:normal
>
Corey, thanks for your suggestion - I'm just now getting the chance
to test with it. In fact, my situation is a little more complicated
than I had let on in my original post. Actually, I have two queues,
xeon and opteron, and about 20 different groups of users. One the
xeon queue, the rey and ce groups should have a priority of 100000, the
greene group should get 10000, and everyone else 1. On the opteron queue,
the nesbitt group should get priority of 20000, 16 other groups in an
"amo" subdivision should get 10000, and everyone else 1. Below is my
maui.cfg which I hoped would set that up.
# maui.cfg 3.2.6p19
SERVERHOST yotta.colorado.edu
# primary admin must be first in list
ADMIN1 root
# Resource Manager Definition
RMCFG[YOTTA.COLORADO.EDU] TYPE=PBS
# Allocation Manager Definition
AMCFG[bank] TYPE=NONE
RMPOLLINTERVAL 00:00:30
SERVERPORT 42559
SERVERMODE NORMAL
# Admin: http://supercluster.org/mauidocs/a.esecurity.html
LOGFILE maui.log
LOGFILEMAXSIZE 10000000
LOGLEVEL 3
# Job Priority: http://supercluster.org/mauidocs/5.1jobprioritization.html
QUEUETIMEWEIGHT 50
PROCWEIGHT 500
BYPASSWEIGHT 1000
# Backfill: http://supercluster.org/mauidocs/8.2backfill.html
BACKFILLPOLICY BESTFIT
RESERVATIONPOLICY CURRENTHIGHEST
BACKFILLMETRIC PROCS
# Node Allocation: http://supercluster.org/mauidocs/5.2nodeallocation.html
NODEALLOCATIONPOLICY MINRESOURCE
# QOS: http://supercluster.org/mauidocs/7.3qos.html
QOSCFG[reyxeon] PRIORITY=100000
QOSCFG[greenexeon] PRIORITY=10000
QOSCFG[normalxeon] PRIORITY=1
QOSCFG[nesbittopt] PRIORITY=20000
QOSCFG[amoopt] PRIORITY=10000
QOSCFG[normalopt] PRIORITY=1
# Creds: http://supercluster.org/mauidocs/6.1fairnessoverview.html
CLASSCFG[xeon] QLIST=reyxeon:greenexeon:normalxeon
CLASSCFG[opteron] QLIST=nesbittopt:amoopt:normalopt
GROUPWEIGHT 1
GROUPCFG[rey] QDEF=reyxeon QLIST=reyxeon:amoopt
GROUPCFG[nesbitt] QDEF=nesbittopt QLIST=nesbittopt:normalxeon
GROUPCFG[kapteyn] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[ce] QDEF=reyxeon QLIST=reyxeon:amoopt
GROUPCFG[anderson] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[bohn] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[cornell] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[cundiff] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[greene] QDEF=greenexeon QLIST=greenexeon:amoopt
GROUPCFG[holland] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[jimenez] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[jin] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[lehnert] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[lewandow] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[lineberg] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[parson] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[perkins] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[thompson] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[weber] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[wieman] QDEF=amoopt QLIST=amoopt:normalxeon
GROUPCFG[ye] QDEF=amoopt QLIST=amoopt:normalxeon
However, it doesn't seem to be assigning group priorities right. When I
submit a job as a member of the ce group to the opteron and xeon queues,
requesting 10 nodes with 4 processors each, here's what happens:
[root at yotta ~]# qstat -an
yotta.colorado.edu:
Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
-------------------- -------- -------- ---------- ------ ----- --- ------ ----- - -----
9682.yotta.colorado. ruprech opteron test -- 10 -- -- -- Q --
--
9685.yotta.colorado. ruprech xeon test -- 10 -- -- -- Q --
--
9686.yotta.colorado. ruprech xeon test -- 10 -- -- -- Q --
--
(Other jobs edited out for clarity.)
[root at yotta ~]# diagnose -p
diagnosing job priority information (partition: ALL)
Job PRIORITY* Cred(Group) Serv(QTime:Bypas) Res( Proc)
Weights -------- 1( 1) 1( 50: 1000) 1( 500)
9682 22138 0.0( 0.0) 9.7(138.3:2000.) 90.3(20000)
9685 22054 0.0( 0.0) 9.3( 54.2:2000.) 90.7(20000)
9686 22050 0.0( 0.0) 9.3( 50.0:2000.) 90.7(20000)
Percent Contribution -------- 0.0( 0.0) 9.4( 0.4: 9.1) 90.6( 90.6)
[root at yotta ~]# diagnose -Q
QOS Status
System QOS Settings: QList: DEFAULT (Def: DEFAULT) Flags: 0
Name * Priority QTWeight QTTarget XFWeight XFTarget QFlags JobFlags Limits
DEFAULT 0 0 0 0 0.00 [NONE] [NONE] [NONE]
[ALL] 0 0 0 0 0.00 [NONE] [NONE] [NONE]
reyxeon 100000 0 0 0 0.00 [NONE] [NONE] [NONE]
Groups: rey ce
Classes: xeon
greenexeon 10000 0 0 0 0.00 [NONE] [NONE] [NONE]
Groups: greene
Classes: xeon
normalxeon 1 0 0 0 0.00 [NONE] [NONE] [NONE]
Groups: nesbitt perkins lewandow bohn weber parson wieman jin jimenez thompson kapteyn cundiff ye anderson holland cornell lineberg lehnert
Classes: xeon
nesbittopt 20000 0 0 0 0.00 [NONE] [NONE] [NONE]
Groups: nesbitt
Classes: opteron
amoopt 10000 0 0 0 0.00 [NONE] [NONE] [NONE]
Groups: perkins lewandow bohn weber parson wieman jin jimenez thompson kapteyn cundiff ye anderson holland cornell lineberg rey ce lehnert greene
Classes: opteron
normalopt 1 0 0 0 0.00 [NONE] [NONE] [NONE]
Classes: opteron
Any ideas what I'm doing wrong here?
Thanks!!
Peter
More information about the mauiusers
mailing list