[torquedev] Feedback on per vnode cpusets

Chris Samuel csamuel at vpac.org
Sun May 25 21:53:01 MDT 2008

----- "Gareth Williams" <Gareth.Williams at csiro.au> wrote:

> Hi All,

Hi Gareth,

> A 4th option is to choose whether or not to use the vnode cpusets on
> a per-job or per-process basis.

There's always one.. :-)

> I think tm_spawn will be the principle (only!) method of
> distributing processes to nodes/cpustes and it could
> honor an environment variable or a property associated
> with the job.

I don't know enough about the tm_spawn functionality to
know how easy that would be.
> This would be in combination with 1), but I would tend to favour
> having the vnode cpusets available but not usually using them.  

I guess a lot depends on what your default MPI implementation
is, for us now having moved to OpenMPI on AMD64 we wouldn't
want anyone to be able to select it in case they manage to
shoot themselves in the foot.

> Note. If (the vnode cpusets were not available but)
> the job cpuset was user writeable, the user could
> construct their own sub-cpusets to contain processes,
> but it would be hard work.

Hmm, I was going to say that sounds like a good plan
but I'm wondering if there's a possible DoS in there,
for instance - I wonder what happens if you request all
the CPUs on a box (that doesn't have a boot cpuset) and
then create a sub-cpuset through that mechanism and mark
it as exclusive ?

Will it let you ? If so, where do all the other processes run ?

Christopher Samuel - (03) 9925 4751 - Systems Manager
 The Victorian Partnership for Advanced Computing
 P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency

More information about the torquedev mailing list