[torquedev] cpuset support

Garrick Staples garrick at usc.edu
Mon Nov 12 17:17:57 MST 2007


I just bumped into Chris Samuel at his (rather barren) booth here at SC07 and I think we just designed cpuset support.

Here's what we came up with...

On startup, pbs_mom will create /dev/cpuset/torque (with all cpus) if it
doesn't already exist and move itself to it.  This allows the admin to stuff
pbs_mom inside a smaller cpuset if desired by creating it in the initscript.
We will call this the "torqueset".

When a job starts, pbs_mom will create a per-job cpuset under the torqueset
with the correct cpus called the "jobset".  It will do this after prologue,
which allows the admin to pre-create it if desired.  This happens on all nodes.

Also, per-vnode cpusets will also be created under the jobset at job start.

pbs_mom will run the batch script inside of the jobset and all TM spawn
requests will run in the vnodeset.

You end up with cpusets that look like:
  /dev/cpuset/torque/job-123.pbsserver.foo.edu/vnode-4

Job exit will consist of ensuring the cpusets are empty (killing processes)
before removing them.

Exclusive cpusets can't be used because of suspended jobs.

All mems will be added to all cpusets unless someone comes up with another idea.

This seems pretty simple to implement, doesn't require any build deps, and
makes sense to me.  Any thoughts?


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/torquedev/attachments/20071112/6bee9417/attachment.bin


More information about the torquedev mailing list