[torqueusers] PBS nodes:ppn

[Troy Baer]

On Thu, 2009-02-12 at 17:10 -0500, Samir Khanal wrote:
> I have been trying this for 3 hours on different combinations but no success.
> I have 6 quad cores and i need to run a prog on 1 core per compute node
> 
> i tried 
> 
> #PBS -l nodes=4:ppn=1
> cat `$PBS_NODEFILE`
> 
> the program runs on a single node utilizing all the cores.
> 
> I am using MPIEXEC
> 
> my submission file looks like
> 
> #PBS -l walltime=1:00:00
> #PBS -N my_job
> #PBS -j oe
> #PBS -l nodes=4:ppn=1
> echo `hostname`
> #echo Directory is `pwd`
> #echo This job is running on following Processors
> echo `cat $PBS_NODEFILE`
> time mpiexec ./bgtw1
> 
> 
> My server_priv/nodes file looks like
> 
> compute-0-0 np=4
> compute-0-1 np=4
> compute-0-2 np=4
> compute-0-3 np=4
> compute-0-4 np=4
> compute-0-5 np=4
> comet       np=8
> 
> #The program always runs on compute-0-5 

What scheduler are you using?  Some of them (e.g. Maui) will assume that
the nodes/ppn geometry can be reshaped by default.

If you're using the OSC mpiexec, something else that may do what you
want is:

#PBS -l walltime=1:00:00
#PBS -N my_job
#PBS -j oe
#PBS -l nodes=4:ppn=1
echo `hostname`
#echo Directory is `pwd`
#echo This job is running on following Processors
echo `cat $PBS_NODEFILE`
time mpiexec -npernode 1 ./bgtw1

This is especially useful if each MPI process is going to spawn threads.

> One more question:
> How do enable the submit maching (comet in my case) to execute jobs?
> I have enabled pbs_mom but it still appears "down"
> 
> comet
>      state = down
>      np = 8
>      ntype = cluster

Is there a running pbs_mom on that host?  If so, what do the log files
say?

	--Troy
-- 
Troy Baer, HPC System Administrator
National Institute for Computational Sciences, University of Tennessee
http://www.nics.tennessee.edu/
Phone:  865-241-4233