[torqueusers] PBS nodes:ppn

pat.o'bryant at exxonmobil.com pat.o'bryant at exxonmobil.com
Fri Feb 13 06:09:26 MST 2009

     Recently we discovered an obscure variable that is part of Torque. It
is the "task per node" variable, or "tpn". Our Moab is configured to
allocate "tasks" not "nodes". I believe Maui has similar settings. What
this means is that a request that looks like "#PBS -l nodes=4,ppn=1", is
translated by Moab to mean a total of "4 tasks". If there is an available
node with 4 processors the job will run on a single node.
     To make it work as desired we code "#PBS -l nodes=4,ppn=1,tpn=1". This
means a total of 4 tasks with 1 task per node. The addition of this
variable has given us the behavior we were looking for. Hopefully this
parameter will work for you as well.


J.W. (Pat) O'Bryant,Jr.
Business Line Infrastructure
Technical Systems, HPC

             Samir Khanal                                                  
             <skhanal at bgsu.                                                
             edu>                                                       To 
             Sent by:                 "torqueusers at supercluster.org"       
             torqueusers-bo           <torqueusers at supercluster.org>       
             unces at superclu                                             cc 
                                      [torqueusers] PBS nodes:ppn          
             02/12/09 04:10                                                

I have been trying this for 3 hours on different combinations but no
I have 6 quad cores and i need to run a prog on 1 core per compute node

i tried

#PBS -l nodes=4:ppn=1

the program runs on a single node utilizing all the cores.

I am using MPIEXEC

my submission file looks like

#PBS -l walltime=1:00:00
#PBS -N my_job
#PBS -j oe
#PBS -l nodes=4:ppn=1
echo `hostname`
#echo Directory is `pwd`
#echo This job is running on following Processors
echo `cat $PBS_NODEFILE`
time mpiexec ./bgtw1

My server_priv/nodes file looks like

compute-0-0 np=4
compute-0-1 np=4
compute-0-2 np=4
compute-0-3 np=4
compute-0-4 np=4
compute-0-5 np=4
comet       np=8

#The program always runs on compute-0-5

One more question:
How do enable the submit maching (comet in my case) to execute jobs?
I have enabled pbs_mom but it still appears "down"

     state = down
     np = 8
     ntype = cluster


torqueusers mailing list
torqueusers at supercluster.org

More information about the torqueusers mailing list