[Mauiusers] Memory resource

Yaroslav Halchenko lists at onerussian.com
Tue Aug 21 11:22:07 MDT 2007

Dear Maui People,

Sorry for a possibly lame question.
We have a cluster running
maui                          3.2.6p20-snap.1176920941-1
torque                        2.1.8-1

before now, most of the tasks were cpu-bound, so I didn't request users
to explicitly request memory resources for their jobs. I've setup
virtual cpus so we have 1 real cpu per queue per node.  Lately we got
more and more memory bound tasks so I am about to implement memory
policy and to request users to provide sensible estimates for the memory
while scheduling the tasks.  But I got surprised by the amounts of RAM
maui is thinking to have per each node.  For instance node25 real amount
of memory (RAM+swap) is 12GB whenever maui thinks 34GB (see screendump
below). The same kind of weird proportion is for the other nodes as

Should memory be specified manually per node via NODECFG in maui.cfg?

Should I just get freshier snapshots?

If I impose memory policy for the tasks (RESOURCELIMITPOLICY
MEM:ALWAYS), how can I prevent users to abuse it and request too much
memory for their tasks, effectively wasting the resources and forbidding other
users to run their tasks even though in reality amount of memory available is
appropriate? Is there a way to punish such strategy (ie comparing
requested and really used memory for the task)?

Thanks everyone in advance for ideas
| itanix:/etc/maui# checknode node25
| checking node node25
| State:   Running  (in current state for 00:00:00)
| Expected State:     Idle   SyncDeadline: Sat Oct 24 08:26:40
| Configured Resources: PROCS: 6  MEM: 34G  SWAP: 34G  DISK: 1M
| Utilized   Resources: [NONE]
| Dedicated  Resources: PROCS: 2  MEM: 1536M
| Opsys:         linux  Arch:      [NONE]
| Speed:      1.00  Load:       2.150
| Network:    [DEFAULT]
| Features:   [matlab][matlab5]
| Attributes: [Batch]
| Classes:    [long 2:2][verylong 0:2][medium 2:2][small 2:2][rumbalong 2:2][rumba 2:2][directors 2:2][rumbasvm 2:2][default 2:2]
| Total Time:   INFINITY  Up:   INFINITY (99.24%)  Active:   INFINITY (59.04%)
| Reservations:
|   User 'SYSTEM.5'(x1)  -16:59:38 ->   INFINITY (  INFINITY)
|     Blocked Resources at 00:00:00    Procs: 6/6 (100.00%)
|   Job '377257'(x1)   -INFINITY -> 58:02:42:41 (83:08:00:00)
|   Job '377898'(x1)  -9:10:55:42 -> 73:21:04:18 (83:08:00:00)
| JobList:  377257,377898
| ALERT:  node is overcommitted at time 00:00:00 (P: -2)
| ALERT:  node is overcommitted at time 58:02:42:41 (P: -1)
| itanix:/etc/maui# pbsnodes node25 | grep totmem
|      status = opsys=linux,uname=Linux node25 2.6.18-4-amd64 #1 SMP Mon Mar 26 11:36:53 CEST 2007 x86_64,sessions=2744 3530 12613 19528 6598 14880 17067,nsessions=7,nusers=5,idletime=957600,totmem=12354868kb,availmem=5005220kb,physmem=6057428kb,ncpus=2,loadave=2.07,netload=121060477190,state=free,jobs=377257.itanix.ravana.rutgers.edu 377898.itanix.ravana.rutgers.edu,rectime=1187716369

Yaroslav Halchenko
Research Assistant, Psychology Department, Rutgers-Newark
Student  Ph.D. @ CS Dept. NJIT
Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
        101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
WWW:     http://www.linkedin.com/in/yarik        

More information about the mauiusers mailing list