[torqueusers] Online docs missing queue resource "nodes"
csamuel at vpac.org
Thu Jan 5 16:44:20 MST 2006
On Friday 06 January 2006 05:01, Martin Siegert wrote:
> with JOBNODEMATCHPOLICY unset:
> 1) "-l nodes=n" requests n processors (not nodes!) anywhere on the cluster
> regardless of whether the processors are on the same node or not.
> (however, if you request "-l nodes=n" with n larger than the no. of
> nodes in your cluster, but smaller than the no. of processors in the
> cluster, the job is rejected by torque with an error message
> "Job exceeds queue resource limits". E.g., you cannot request
> -l nodes=6 on a cluster with 4 dual processor nodes. I suspect that
> this is a bug).
You can override this as I bugged David et. al about this for ages and they
gave in to make me go away. :-)
So for our 144 CPU Power5 cluster (36 x 4 CPU boxes) I have:
set server resources_available.nodect = 144
and Torque no longer complains when I ask for nodes=n where 0 < n < 145.
In other words with that set I can use nodes purely as a request for a number
of CPUs in any arrangement.
> 2) "-l nodes=n:ppn=1" works in exactly the same way as 1, i.e., you may
> actually get two processors on the same node. Similarly,
> "-l nodes=n:ppn=m" means "give me n*m processors with at least m
> processors on each node".
I think the behaviour is slightly different (though this may be because we're
using Moab rather than Maui).
I believe that you will get n*m CPUs with at *least* m per node, though you
may have some combination that gives you more than m on some or all nodes.
However, we never get less than 'm' per node, so again on our Power5 cluster a
user can ask for nodes=1:ppn=4 and be guaranteed to get a single node to
themselves for an SMP code or some other reason.
> 3) "-l ncpus=n" requests n processors on a single node, i.e., the same
> as "-l nodes=1:ppn=n"
IIRC this has caused interesting behaviours on the rare occasions when one or
two of our users have tried that, and we never use it ourselves.
Christopher Samuel - (03)9925 4751 - VPAC Deputy Systems Manager
Victorian Partnership for Advanced Computing http://www.vpac.org/
Bldg 91, 110 Victoria Street, Carlton South, VIC 3053, Australia
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/torqueusers/attachments/20060106/1369e631/attachment.bin
More information about the torqueusers