[torqueusers] procs resource advice and docs
siegert at sfu.ca
Wed Jan 20 16:30:36 MST 2010
just to expand a little bit on what Roman already said ...
On Wed, Jan 20, 2010 at 09:55:12PM +1100, Gareth.Williams at csiro.au wrote:
> Hi All,
> After finding out that there is a 'procs' resource a while back, I've run
> some tests (with a 2.3 torque and moab) specifying procs instead of nodes
> and think I'd like to recommend it to our user community (particularly on
> our 128p numa system). It complements the 'nodes' resource specification
> nicely for when you just want allocated cores and don't care about the
> how they are arranged on nodes.
> Is anybody already using the feature?
We are using this together with the
in moab. This way the -l nodes=n:ppn=m specification is interpreted the
way users expect it to work and at the same time we have a method available
to users who do not care how their processes are distributed accross nodes.
After gaining some experience with this we now recommend to users to
use -l procs=N unless they have a specific reason to use the -l nodes=n:ppn=m
syntax: the waiting time in the queue with -l procs=N is much, much
> Is there a downside apart from the lack of documentation?
None. Only benefits. The usage percentage of the cluster increases
> I see it's documented in the pbs_resources_unicos8 man page so I guess
> it was developed by or for CRAY, possibly long ago, but it seems to work
> fine on our linux systems. The pbs_resources_unicos8 man page does not
> mention that it conflicts with the nodes resource syntax but I guess this
> is obvious.
Actually, -l procs is fairly new - we requested that feature :-)
Thus, I do not believe that whatever is mentioned under CRAY applies.
> moab just uses the nodes resource request if it is present (ignoring
> the procs resource)
Actually it is the other way round: if you specify procs then nodes are
(I am confused here: that page existed a few days ago, but now it is gone).
> and I guess maui would too but that is a function of the scheduler
> and doesn't really have a place in the torque docs per-se.
I am not sure whether maui has support for procs.
> Can the linux pbs_resources man page should be updated to include
> the procs resource?
There is one more issue: if you are using OSC's mpiexec you need
to patch the code in get_hosts.c to add support for procs. I can
email you the patch, if you need it.
Head, Research Computing
WestGrid Site Lead
IT Services phone: 778 782-4691
Simon Fraser University fax: 778 782-4242
Burnaby, British Columbia email: siegert at sfu.ca
Canada V5A 1S6
More information about the torqueusers