[torquedev] [Bug 93] Resource management semantics of Torque need to be well defined

bugzilla-daemon at supercluster.org bugzilla-daemon at supercluster.org
Thu Oct 28 03:56:58 MDT 2010


--- Comment #2 from Simon Toth <SimonT at mail.muni.cz> 2010-10-28 03:56:57 MDT ---
> External schedulers - I think you're right for both Moab and Maui, they both
> set exec_host.

That would be great.

> PPN = processors per node (according to manual page), really virtual processors
> as you can overcommit if you are not using cpusets.  I've seen plenty of
> commercial software out there that uses them, so I don't think it can go away. 
> The pvmem limits which you mention are vital to us.

Well, that's the problem, then manual page says processors per node, but that's
not how Torque works (this is exactly the reason why I created this bug). They
are processes per node. I'm not saying to get rid of ppn, but to get rid of the
processes semantics, therefore ppn will be actually processors not processes.
pvmem can actually stay, although I think pmem and pvmem can be easily
superseded by mem and vmem.

Plus when you request -l nodes=2:ppn=2:pvmem=2G how much memory do you expect
to get? In the current Torque semantics it is 2*2*2G.

> Different resource limits - I think the current per process and per job limits
> make enough sense, it's easy for users to understand.  The only real issue is
> that you cannot set a proactively enforced (i.e. malloc fails) limit across a
> job as a whole.  But that's enforced by the scheduler anyway (at least with
> Maui and Moab).

The issue is that it is enforced internally by the schedulers. My target is to
make all this work even with qrun. That implies that basic resources like mem,
cpus, etc.. must have a well defined semantic inside Torque itself.

> Cgroups - I reckon it's a good plan for the future but we need to realise that
> it's not going to really arrive for most clusters until RHEL6/CentOS6 starts
> getting deployed. Also you cannot have both cpusets and cgroups mounted at the
> same time so the current code needs to be refactored/abstracted to be able to
> cope with either one being present.
> It cannot depend on a feature of cgroups being present but should give you the
> benefits if it is.

Actually my idea was to create a new cgroups platform (new folder in

Configure bugmail: http://www.clusterresources.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

More information about the torquedev mailing list