[Mauiusers] per-queue group priority setting

Peter Ruprecht ruprech at jilau1.Colorado.EDU
Wed Jul 23 09:29:53 MDT 2008


On Thu, 3 Jul 2008, Corey Ferrier wrote:

> On Wed, Jul 02, 2008 at 01:21:47PM -0600, Peter Ruprecht wrote:
> >
> >Hi,
> >
> >I'm running maui 3.2 as a scheduler for torque 2.1.8.  Suppose I have
> >two queues, called "xeon" and "opteron".  Also, there are two groups
> >of users, called "smith" and "jones".
> >
> >I'd like both groups to be able to submit jobs to either queue, but for
> >the smith group to have higher default priority on xeon, while jones
> >has higher priority on opteron.
> >
> >Is there some combination of QOSCFG, GROUPCFG, and PRIORITY that will
> >enable this?  For example if I use just
> >
> >GROUPCFG[smith]       PRIORITY=20000
> >
> >there's no indication of which queue it applies to.
> >
> >Thanks for any suggestions,
> >
>
> Would this work?
>
> CLASSCFG[xeon]       QLIST=xeonprio:normal
> CLASSCFG[opteron]    QLIST=opteronprio:normal
>
> QOSCFG[xeonprio]     PRIORITY=20000
> QOSCFG[opteronprio]  PRIORITY=20000
> QOSCFG[normal]       PRIORITY=1
>
> GROUPCFG[smith]      QDEF=xeonprio    QLIST=xeonprio:normal
> GROUPCFG[jones]      QDEF=opteronprio QLIST=opteronprio:normal
>

Corey, thanks for your suggestion - I'm just now getting the chance
to test with it.  In fact, my situation is a little more complicated
than I had let on in my original post.  Actually, I have two queues,
xeon and opteron, and about 20 different groups of users.  One the
xeon queue, the rey and ce groups should have a priority of 100000, the
greene group should get 10000, and everyone else 1.  On the opteron queue,
the nesbitt group should get priority of 20000, 16 other groups in an
"amo" subdivision should get 10000, and everyone else 1.  Below is my
maui.cfg which I hoped would set that up.

# maui.cfg 3.2.6p19
SERVERHOST            yotta.colorado.edu
# primary admin must be first in list
ADMIN1                root
# Resource Manager Definition
RMCFG[YOTTA.COLORADO.EDU] TYPE=PBS
# Allocation Manager Definition
AMCFG[bank]  TYPE=NONE

RMPOLLINTERVAL        00:00:30

SERVERPORT            42559
SERVERMODE            NORMAL

# Admin: http://supercluster.org/mauidocs/a.esecurity.html
LOGFILE               maui.log
LOGFILEMAXSIZE        10000000
LOGLEVEL              3

# Job Priority: http://supercluster.org/mauidocs/5.1jobprioritization.html
QUEUETIMEWEIGHT       50
PROCWEIGHT            500
BYPASSWEIGHT          1000

# Backfill: http://supercluster.org/mauidocs/8.2backfill.html
BACKFILLPOLICY        BESTFIT
RESERVATIONPOLICY     CURRENTHIGHEST
BACKFILLMETRIC        PROCS

# Node Allocation: http://supercluster.org/mauidocs/5.2nodeallocation.html
NODEALLOCATIONPOLICY  MINRESOURCE

# QOS: http://supercluster.org/mauidocs/7.3qos.html
QOSCFG[reyxeon]      PRIORITY=100000
QOSCFG[greenexeon]   PRIORITY=10000
QOSCFG[normalxeon]   PRIORITY=1
QOSCFG[nesbittopt]   PRIORITY=20000
QOSCFG[amoopt]       PRIORITY=10000
QOSCFG[normalopt]    PRIORITY=1

# Creds: http://supercluster.org/mauidocs/6.1fairnessoverview.html

CLASSCFG[xeon]       QLIST=reyxeon:greenexeon:normalxeon
CLASSCFG[opteron]    QLIST=nesbittopt:amoopt:normalopt

GROUPWEIGHT             1
GROUPCFG[rey]        QDEF=reyxeon       QLIST=reyxeon:amoopt
GROUPCFG[nesbitt]    QDEF=nesbittopt    QLIST=nesbittopt:normalxeon
GROUPCFG[kapteyn]    QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[ce]         QDEF=reyxeon       QLIST=reyxeon:amoopt
GROUPCFG[anderson]   QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[bohn]       QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[cornell]    QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[cundiff]    QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[greene]     QDEF=greenexeon    QLIST=greenexeon:amoopt
GROUPCFG[holland]    QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[jimenez]    QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[jin]        QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[lehnert]    QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[lewandow]   QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[lineberg]   QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[parson]     QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[perkins]    QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[thompson]   QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[weber]      QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[wieman]     QDEF=amoopt        QLIST=amoopt:normalxeon
GROUPCFG[ye]         QDEF=amoopt        QLIST=amoopt:normalxeon



However, it doesn't seem to be assigning group priorities right.  When I
submit a job as a member of the ce group to the opteron and xeon queues,
requesting 10 nodes with 4 processors each, here's what happens:

[root at yotta ~]# qstat -an
yotta.colorado.edu:
                                                                   Req'd  Req'd   Elap
Job ID               Username Queue    Jobname    SessID NDS   TSK Memory Time  S Time
-------------------- -------- -------- ---------- ------ ----- --- ------ ----- - -----
9682.yotta.colorado. ruprech  opteron  test          --     10  --    --    --  Q   --
    --
9685.yotta.colorado. ruprech  xeon     test          --     10  --    --    --  Q   --
    --
9686.yotta.colorado. ruprech  xeon     test          --     10  --    --    --  Q   --
    --

(Other jobs edited out for clarity.)


[root at yotta ~]# diagnose -p
diagnosing job priority information (partition: ALL)
Job                    PRIORITY*   Cred(Group)  Serv(QTime:Bypas)   Res( Proc)
             Weights   --------       1(    1)     1(   50: 1000)     1(  500)
9682                      22138     0.0(  0.0)   9.7(138.3:2000.)  90.3(20000)
9685                      22054     0.0(  0.0)   9.3( 54.2:2000.)  90.7(20000)
9686                      22050     0.0(  0.0)   9.3( 50.0:2000.)  90.7(20000)
Percent Contribution   --------     0.0(  0.0)   9.4(  0.4:  9.1)  90.6( 90.6)

[root at yotta ~]# diagnose -Q
QOS Status

System QOS Settings:  QList: DEFAULT (Def: DEFAULT)  Flags: 0

Name                * Priority QTWeight QTTarget XFWeight XFTarget     QFlags   JobFlags Limits

DEFAULT                      0        0        0        0     0.00     [NONE]     [NONE] [NONE]
[ALL]                        0        0        0        0     0.00     [NONE]     [NONE] [NONE]
reyxeon                 100000        0        0        0     0.00     [NONE]     [NONE] [NONE]
  Groups:    rey ce
  Classes:  xeon
greenexeon               10000        0        0        0     0.00     [NONE]     [NONE] [NONE]
  Groups:    greene
  Classes:  xeon
normalxeon                   1        0        0        0     0.00     [NONE]     [NONE] [NONE]
  Groups:    nesbitt perkins lewandow bohn weber parson wieman jin jimenez thompson kapteyn cundiff ye anderson holland cornell lineberg lehnert
  Classes:  xeon
nesbittopt               20000        0        0        0     0.00     [NONE]     [NONE] [NONE]
  Groups:    nesbitt
  Classes:  opteron
amoopt                   10000        0        0        0     0.00     [NONE]     [NONE] [NONE]
  Groups:    perkins lewandow bohn weber parson wieman jin jimenez thompson kapteyn cundiff ye anderson holland cornell lineberg rey ce lehnert greene
  Classes:  opteron
normalopt                    1        0        0        0     0.00     [NONE]     [NONE] [NONE]
  Classes:  opteron


Any ideas what I'm doing wrong here?

Thanks!!
Peter


More information about the mauiusers mailing list