[torqueusers] Torque and MPI jobs

Albino Aveleda bino at coc.ufrj.br
Sat Nov 11 06:45:32 MST 2006


Hi Ana,

If you need the file like this:
n001:4
n002:4
...

you can put this line in your job file:

cat ${PBS_NODEFILE} | uniq | sed 's/$/:4/' > /tmp/${PBS_JOBID}

The "uniq" will remove the equal line and "sed" will add to end line 
":4". You have to use /tmp/${PBS_JOBID} with mpdboot
and the last line I should add this line.

rm -f /tmp/${PBS_JOBID}

Best Regards,
Albino

Quoting Anna Jonna Armannsdottir <annaj at hi.is>:

> On fös, 2006-11-10 at 21:20 -0200, Albino Aveleda wrote:
>> --- script to run with 4 nodes ---
>> #PBS -l nodes=4
>> #PBS -l walltime=06:00:00
>> #PBS -j oe
>> #PBS -N mpi
>
> Hi and thanks for your comments.
> A script like yours runs fine on my setup.
> When I want to use 4 processes on each node the mpd
> complains because it does not understand it.
> Of course I could run it with 1 processor on each node
> but that would be a terrible waste of computing power.
>
> my submit script is like this:
>
> #PBS -N my_submit_pbs
> #PBS -l nodes=4:ppn=4
> #PBS -j oe
> #PBS -V
> #PBS -q short
>
>
> APP_DIR=/usr/local/nw-python-mpi/bin
> BIN_APP=nw
> MPI_APP=${BIN_APP}.mpi
> #APP_PARAM=$HOME/nitrogen.nw
> APP_PARAM=$HOME/neon.nw
>
>
> cd $PBS_O_WORKDIR
> NPROCS=`wc -l < $PBS_NODEFILE`
> echo This jobs runs on the following $NPROCS processors:
> /bin/sort ${PBS_NODEFILE}
> MPI_NODEFILE=$(echo /tmp/$( /bin/basename ${PBS_NODEFILE} ).mpi)
> echo MPI nodefile: ${MPI_NODEFILE}
>
> echo "Job started at: `date`"
>
> if test ${NPROCS} -gt 1
> then
>  touch ${MPI_NODEFILE} && \
> while read name
> do
>  echo $name:4 >> ${MPI_NODEFILE}
>  let "count += 1"
> done <"$PBS_NODEFILE"    # Redirects stdin to file $Filename.
>
> cat ${MPI_NODEFILE}
>  /usr/local/bin/mpdboot --totalnum=4 --file=${PBS_NODEFILE}
>  /usr/local/bin/mpiexec -l -n ${NPROCS} ${APP_DIR}/${MPI_APP}
> ${APP_PARAM}
>  /usr/local/bin/mpdallexit
> elif test ${NPROCS} -eq 1
> then
>  ${APP_DIR}/${BIN_APP} ${APP_PARAM}
> fi
>
> echo "Job finished at: `date`"
>
> --
> Kær kveðja, Anna Jonna Ármannsdóttir,
> Unix Kerfisstjórnun, Reiknistofnun HÍ
>
>




__________________________________________________
Albino A. Aveleda                 bino at coc.ufrj.br
Network Manager                   +55 21 2562-8080
PEC-COPPE/UFRJ                    +55 21 2562-8465
Federal University of Rio de Janeiro (UFRJ)


More information about the torqueusers mailing list