[torqueusers] Online docs missing queue resource "nodes"

[Chris Samuel]

On Friday 06 January 2006 05:01, Martin Siegert wrote:

> with JOBNODEMATCHPOLICY unset:
>
> 1) "-l nodes=n" requests n processors (not nodes!) anywhere on the cluster
>    regardless of whether the processors are on the same node or not.
>    (however, if you request "-l nodes=n" with n larger than the no. of
>    nodes in your cluster, but smaller than the no. of processors in the
>    cluster, the job is rejected by torque with an error message
>    "Job exceeds queue resource limits". E.g., you cannot request
>    -l nodes=6 on a cluster with 4 dual processor nodes. I suspect that
>    this is a bug).

You can override this as I bugged David et. al about this for ages and they 
gave in to make me go away. :-)

So for our 144 CPU Power5 cluster (36 x 4 CPU boxes) I have:

  set server resources_available.nodect = 144

and Torque no longer complains when I ask for nodes=n where 0 < n < 145.

In other words with that set I can use nodes purely as a request for a number 
of CPUs in any arrangement.

> 2) "-l nodes=n:ppn=1" works in exactly the same way as 1, i.e., you may
>    actually get two processors on the same node. Similarly,
>    "-l nodes=n:ppn=m" means "give me n*m processors with at least m
>    processors on each node".

I think the behaviour is slightly different (though this may be because we're 
using Moab rather than Maui).

I believe that you will get n*m CPUs with at *least* m per node, though you 
may have some combination that gives you more than m on some or all nodes.

However, we never get less than 'm' per node, so again on our Power5 cluster a 
user can ask for nodes=1:ppn=4 and be guaranteed to get a single node to 
themselves for an SMP code or some other reason.

> 3) "-l ncpus=n" requests n processors on a single node, i.e., the same
>    as "-l nodes=1:ppn=n"

IIRC this has caused interesting behaviours on the rare occasions when one or 
two of our users have tried that, and we never use it ourselves.

cheers,
Chris
-- 
 Christopher Samuel - (03)9925 4751 - VPAC Deputy Systems Manager
 Victorian Partnership for Advanced Computing http://www.vpac.org/
 Bldg 91, 110 Victoria Street, Carlton South, VIC 3053, Australia

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/torqueusers/attachments/20060106/1369e631/attachment.bin