[torqueusers] vmem and pvmem

[Martin Siegert]

Hi,

I am struggling with the implmentation of the vmem and pvmem
resources (in principle the exact same concerns apply to mem
and pmem). Let's say I set

resources_default.pvmem = 1gb

in qmgr. Now a user submits a job requesting -l procs=1,vmem=2gb
and the job fails because it exceeds the pvmem default. Apparently
torque treats vmem and pvmem as two independent resources, which
(in particular for 1p jobs) is not very reasonable.
Similarly, if I would set resources_default.vmem, jobs that
request pvmem fail even if the specified
(amount of pvmem)*(no. of requested processors) > vmem default

How do people deal with this issue?
As far as I can tell moab only uses pmem and pvmem, i.e., moab
converts vmem to pvmem = vmem/procct and mem to vmem = mem/procct.
Correct? Shouldn't torque do the same?
I am worried about shared memory jobs though - jobs were pvmem
is not really relevant since all processes share the same memory
and vmem is not simply the sum of the process memory usage (at
least you cannot add up amounts displayed by ps, etc.). But I do
not know whether torque handles this correctly anyway, does it?

For now I modified the torque_submitfilter posted by Gareth
http://www.clusterresources.com/pipermail/torquedev/2011-March/003479.html
(thanks Gareth!) to add a qsub option -l pvmem=... in those
cases when the user requests vmem, but this appears to be an ugly
workaround. Shouldn't there be a better way?

Cheers,
Martin

-- 
Martin Siegert
Head, Research Computing
Simon Fraser University
Burnaby, British Columbia