[torqueusers] Memory resource Limits
csamuel at vpac.org
Sun Aug 3 22:06:52 MDT 2008
----- "Jim Kusznir" <jkusznir at gmail.com> wrote:
> We've tried mem= with no results; pvmem resulted
> in killing all jobs unless it was actually using
> an extremely small amount of ram.
Whilst PBS does monitor a job to try and work out
when it's breaking the set limits it is also complicated
by ulimits and glibc's malloc() I'm afraid.
By default Torque sets two different ulimits depending
on what you request.
If you set a mem or pmem limit then pbs_mom sets the RLIMIT_DATA
size which is described as:
The maximum size of the process’s data segment (initialized
data, uninitialized data, and heap). This limit affects calls
to brk(2) and sbrk(2), which fail with the error ENOMEM upon
encountering the soft limit of this resource.
If you set vmem or pvmem then it sets the RLIMIT_AS size
which is described as:
The maximum size of the process’s virtual memory (address space)
in bytes. This limit affects calls to brk(2), mmap(2) and
mremap(2), which fail with the error ENOMEM upon exceeding this
Notice that the first only affects brk()/sbrk() whilst
the second affects mmap() et. al as well as brk() ?
The kicker is that glibc's malloc() will switch between
using brk() and mmap() depending on the size of allocation
that you're trying to make. So if you request pmem and
your program makes large allocations it will never hit
RLIMIT_DATA as glibc will be making calls to mmap().
Christopher Samuel - (03) 9925 4751 - Systems Manager
The Victorian Partnership for Advanced Computing
P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency
More information about the torqueusers