[torqueusers] NCPUS environment variable?

Martin Siegert siegert at sfu.ca
Mon Jul 11 18:12:02 MDT 2005


Hi,

I am wondering whether (how?) the number of processors for a MPI
job is available in a PBS script in an environment variable (or
otherwise).

Example: I have a script

#!/bin/bash
#PBS -S /bin/bash
#PBS -l walltime=0:12:00
#PBS -l nodes=4:ppn=2
mpiexec -machinefile $PBS_NODEFILE -np 8 ./mpi_program

I would prefer having an environment variable (e.g., PBS_NCPUS) set by
torque so that I could use

#!/bin/bash
#PBS -S /bin/bash
#PBS -l walltime=0:12:00
#PBS -l nodes=4:ppn=2
mpiexec -machinefile $PBS_NODEFILE -np $PBS_NCPUS ./mpi_program

Similarly on a shared memory machine I would like to use

#!/bin/bash
#PBS -S /bin/bash
#PBS -l walltime=0:12:00
#PBS -l ncpus=8
mpiexec -machinefile $PBS_NODEFILE -np $PBS_NCPUS ./mpi_program

Is this possible? I found that torque provides $PBS_NODENUM and $PBS_TASKNUM
but those always seem to get set to 0 and 1, resp. Am I missing something?

On a cluster I can cound the number of lines in $PBS_NODEFILE, but on
a IBM shared memory machine $PBS_NODEFILE only contains a single line
regardless of how many processors were requested in the
#PBS -l ncpus=...
line.

If such an environment variable does not exist alread, could this be
implemented in a future version? I briefly looked at the source, but
it was not obvious to me which routine should set that variable.

On a somewhat unrelated topic:
On a cluster with dual processor nodes is it possible to request 8 cpus
without specifying the number of processors per node? I do not care
whether I get 2 processors per node or just one (or a mix), I just want
to get 8 processors in total as soon as they become available anywhere
on the cluster. Since we have quite a few serial jobs of various
runtimes it is much easier to get 8 cpus (e.g., 2 processors on nodes
a and b, and 1 processors each on nodes c,d,e,f) than 4 nodes with
2 processors each (or 8 nodes with one processor each).
Ideally, I would like to use
#PBS -l ncpus=8
on a cluster as well - in that case I would not have to change the
submission scripts between the different architectures.

Cheers,
Martin

-- 
Martin Siegert
Head, HPC at SFU
WestGrid Site Manager
Academic Computing Services                        phone: (604) 291-4691
Simon Fraser University                            fax:   (604) 291-4242
Burnaby, British Columbia                          email: siegert at sfu.ca
Canada  V5A 1S6


More information about the torqueusers mailing list