[torquedev] [PATCH] Change pbs_mom to set RLIMIT_AS instead of
RLIMIT_DATA for mem/pmem limits.
ake.sandgren at hpc2n.umu.se
Sun Jan 11 02:10:05 MST 2009
On Sun, 2009-01-11 at 14:07 +1100, David Singleton wrote:
> Chris Samuel wrote:
> > ----- "David Singleton" <David.Singleton at anu.edu.au> wrote:
> > Hi Dave!
> >> Hi Chris,
> >> I have to agree with Ake:
> >> * mem/pmem is for limiting RSS/physical memory use
> >> * vmem/pvmem is for limiting virtual memory/address ranges
> > But that's not what pbs_mom is designed to do,
> Hmmm, well it does in OpenPBS. Looks like Torque broke this at some
> stage. But the Torque doco still describes how it should be:
It's not really broken in Torque. It still sets _RSS for pmem but it
also sets DATA and STACK for some reason.
> mem Maximum amount of physical memory used by the job.
> pmem Maximum amount of physical memory used by any single process of the job.
> pvmem Maximum amount of virtual memory used by any single process in the job.
> vmem Maximum amount of virtual memory used by all concurrent processes in the job.
> How did RLIMIT_DATA get into limiting "mem"?
Not sure. I don't have anything older then 1.2.0p6 of torque available
and that already contained my patch for the change to use _AS for pvmem.
I don't have my original patch around any more.
In the current situation with the kernel not enforcing _RSS one should
use pvmem and not pmem as the limiting factor for memory.
There might be a way to solve this if the kernel starts using memorysets
(like cpusets) but i haven't seen any real plans yet.
To make things more complicated there are situations where one would
like to have different pvmem limits on different nodes in a job.
i.e. nodes=1+2:ppn=8 with pvmem=16g+16:2g meaning 1 node with 16g and 2
nodes with 8 cores each with 2g.
So this resource limit setup really needs a good overhaul.
Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden
Internet: ake at hpc2n.umu.se Phone: +46 90 7866134 Fax: +46 90 7866126
Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se
More information about the torquedev