[torqueusers] qsub and mpiexec -f machinefile

Michel Béland michel.beland at calculquebec.ca
Thu Feb 20 06:09:50 MST 2014

Tiago Silva (Cefas) wrote:

> Thanks, this seems promising. Before I try building with openmpi, if I 
> parse PBS_NODEFILE to produce my own machinefile for mpiexec, for 
> instance following my previous example:
> n100
> n100
> n101
> n101
> n101
> n101
> won't mpiexec start mpi processes with ranks 0-1 onto n100 and with 
> rank 2-5 on n101? That what I think it does when I don't use qsub.

Yes, but you should not change the nodes inside the $PBS_NODEFILE. You 
can change the order but do not delete machines and add new ones, 
otherwise your MPI code will try to run on nodes belonging to other jobs.

If you want to have exactly the nodes above, you can ask for 
-lnodes=n100:ppn=2+n101:ppn=4. If you only want two cores on the first 
node and four on the second but the specific nodes are irrelevant, you 
can ask for -lnodes=1:ppn=2+1:ppn=4 instead.

Michel Béland
Calcul Québec

More information about the torqueusers mailing list