[torqueusers] vmem and pvmem
"Mgr. Šimon Tóth"
toth at fi.muni.cz
Sat Feb 25 03:18:30 MST 2012
On 25.2.2012 00:50, David Singleton wrote:
> On 02/25/2012 09:00 AM, Martin Siegert wrote:
>> On Fri, Feb 24, 2012 at 11:19:37AM +0100, "Mgr. Šimon Tóth" wrote:
>>>> Core_req vmem pvmem ulimit-v RPT
>>>> nodes=1:ppn=2 1gb 256mb 256mb 512mb
>>>> procs=2 1gb 256mb 256mb 1gb
>>>> nodes=1:ppn=2 1gb 4gb 1gb 4gb
>>>> procs=2 1gb 4gb 1gb 4gb
>>>> nodes=1:ppn=2 1gb - 1gb 512mb
>>>> procs=2 1gb - 1gb 1gb
>>>> So the ulimit value that influences whether a task can allocate
>>>> memory, is set as the lower of the vmem and pvmem values. That
>>>> makes some sense - at least more sense than taking the larger
>>>> value. What doesn't make sense is allowing pvmem to be higher
>>>> than vmem in the first place - in that case torque should probably
>>>> reject the job or 'fix' one of the settings but leaving it as is
>>>> might not be so bad, except for moab's behaviour (keep reading).
>>> No. The logic is as follows:
>>> * if pvmem (or pmem) is set
>>> then set the corresponding ulimit to pvmem (pmem) value
>>> * if pvmem (or pmem) isn't set
>>> then set the corresponding ulimit to vmem (mem) value
>>> Note that using pvmem is mostly pointless. On Linux this represents
>>> address space, not virtual memory.
>>> You can use vmem as virtual memory, but even that is extremely confusing.
>> I do not understand this comment. Both pvmem and vmem requests will
>> result in RLIMIT_AS getting set.
> I disagree with vmem setting RLIMIT_AS if that is what is happening.
Yes it does. It is logical, but also pointless, since this value is
checked by the node itself. Therefore if the process is not killed by
the ulimit it will be killed by the node daemon.
The issue here is that RLIMIT_AS is limiting address space (on Linux).
That's not a resource. If you map a 1GB file to memory your are not
using up anything.
The correct resource to be limited is mem (pmem) not vmem (pvmem).
Mgr. Simon Toth
More information about the torqueusers