[torqueusers] PBS_NODEFILE order question

[Jackie Scoggins]

torque - version 2.1.8
maui version 3.2.6p19

Is there a way to make the PBS_NODEFILE order the nodes as follows:

n0000
n0001
n0000
n0001
n0000
n0001

instead of in order like:

n0000
n0000
n0000
n0000
n0001
n0001
n0001
n0001

based on your request of

-l nodes=2:ppn=4

The reason for this request is so that we would like users to be able  
to specify that they want to run on less processors than are available  
per node for memory intensive jobs.  If you are allocated 8 processors  
(on two nodes as above) and you naively request mpirun -np 4 then you  
will end up running only on the first node, this defeats the purpose.

The current situation does permit ppn=1 to be specified, however, this  
still allows another job to run on the same nodes, so that too is  
undesirable.

I have tried chopping up the PBS_NODEFILE and using a different  
hostfile for mpirun but I get an orted error from mpirun because it  
does not look like it is selecting the nodes in the order in which I  
specify in my hostfile.

Any help on this will be appreciated.

Thanks

Jackie