[torqueusers] Using procs and grouping by even numbers of processors/node

Martin Siegert siegert at sfu.ca
Thu Oct 27 17:48:01 MDT 2011

Hi Lance,

On Thu, Oct 27, 2011 at 10:57:00AM -0400, Lance Westerhoff wrote:
> Hello All-
> Another question that has come up in the use of the procs qsub argument.
> When you run GAMESS, apparently you need to group the processors evenly
> among nodes. In other words, if you ask for procs=4 and 3 processors are
> allocated on one node and 1 allocated on the other, the job will fail
> since GAMESS requires that an even number of processes be available on
> the nodes on which it runs. In this case, the job would have been
> successful if 2 processes were started on one node and 2 processes
> were started on the other. Is there anyway to modify or augment the
> procs option in order to make sure that an even number of processes are
> started on each node?
> Thanks!
> -Lance
> ____________________
> Lance M. Westerhoff, Ph.D.
> General Manager
> QuantumBio Inc. 
> WWW:    http://www.quantumbioinc.com
> Email:    lance at quantumbioinc.com
> Phone:   814-235-6908
> Fax:        814-235-6909

Actually, GAMESS prefers for best overall performance to run two processes
on the same core since the data process is mostly idle (i.e., running
the data process on a core by itself is a waste of resources). Thus, the
following should work:

#PBS -l procs=15

rm -f myhosts.$PBS_JOBID
while read line; do
   echo $line >> myhosts.$PBS_JOBID
   echo $line >> myhosts.$PBS_JOBID
NP=`expr $PBS_NP \* 2`
mpiexec -n $NP -hostfile myhosts.$PBS_JOBID gamess.x

You probably want to use cpusets when doing things like this.


Martin Siegert
Head, Research Computing
Simon Fraser University

More information about the torqueusers mailing list