[Mauiusers] Maui ignores consumable resources

[Jörg Bornschein]

Hello,


I'm having some trouble configuring node-locked consumable resources. We
are running maui-3.2.6p21 with torque-2.3.7 on Ubuntu Jaunty. Both
compiled locally, not using precompiled packages.


I configured maui.conf to let it know some of our nodes have CUDA
eneabled GPUs available:

-- maui.cfg --
[..]

NODECFG[cn00]  GRES=cuda:4
NODECFG[cn01]  GRES=cuda:4
----

It seems maui is aware of this resource -- in it's logfile I find:

--- maui.log | grep cuda ---
07/24 12:35:22 INFO:     NODECFG[0] set to GRES=cuda:4
07/24 12:35:22 INFO:     adding MAList[GRes][1]: 'cuda'
07/24 12:35:22 INFO:     NODECFG[0] set to GRES=cuda:4
---

and checknode cn00 returns:

--- checknode cn00 ---
checking node cn00

State:      Idle  (in current state for 00:01:33)
Configured Resources: PROCS: 8  MEM: 15G  SWAP: 34G  DISK: 1M  cuda: 4
Utilized   Resources: cuda: 4
Dedicated  Resources: [NONE]
Opsys:         linux  Arch:      [NONE]
Speed:      1.00  Load:       0.120
Network:    [DEFAULT]
Features:   [NONE]
Attributes: [Batch]
Classes:    [test1 8:8][full_nodes 8:8][day_queue 8:8]

[..]
---

(opposed to "checknode cn16", where neither the Configured nor the
utilized resources show cuda)


The problem is, that jobs that request "cuda" are scheduled everywhere,
even on nodes without this resource. If, for example I run

 qsub -l gres=cuda -I

my job is started on cn16. I put the job-launch-time maui.log to

 http://fias.uni-frankfurt.de/~bornschein/maui-log1.txt

I also tried the qsub -W x="GRES:cuda" syntax, but it did not change the
result.


Any advice on how to proceed?



thanks



  joerg