[torqueusers] Success setting up a new Torque Environment in University

Gus Correa gus at ldeo.columbia.edu
Fri Sep 2 07:46:51 MDT 2011


Hi Guilherme

To add to Ken's suggestions.

1) Make sure on the [head] node where you run pbs_server
your ${TORQUE}/server_priv/nodes file describes correctly
your cluster configuration.
Something like this [minimally, there are more options]:
node01 np=8
node02 np=8
...
node23 np=8
(Assuming the nodes's hostnames are node01, etc, and have
8 cpu cores each. Adjust to your reality.)

If not, edit that file, and restart your pbs_server.

2) To test, as Ken said, use pbsdsh.

See the pbsdsh man pages.
Check also the qsub man page.

Although you can pass the job parameters to qsub via command line,
you may want to write a little script:
#PBS -q batch
#PBS -l nodes=4:ppn=8
#PBS -n my_job
...
mpiexec -np 32 ./my_mpi_program  [or the pbsdsh command]

and do 'qsub my_pbs_script'

2) I am not familiar to clustalw, which seems to be a genome
sequencing program.
Is it parallel?
Does it use MPI?
Does it require a specific flavor of MPI.

If it works with any MPI, you could install MPICH2 (if your network
is Gigabit Ethernet), MVAPICH2 (if your network is Infiniband),
or OpenMPI (for Gigabit Ethernet, or Infiniband, or Myrinet).
All of them can be built with the Gnu compilers (gcc,g++,gfortran),
or with commercial compilers (Intel, PGI, etc).

See:
http://www.open-mpi.org/
http://www.mcs.anl.gov/research/projects/mpich2/
http://mvapich.cse.ohio-state.edu/overview/mvapich2/

I am fond of OpenMPI because it is very flexible and has excellent
integration to Torque.

Then you need to build clustall linking it to your MPI.
Typically this is easier to do using the the MPI compiler wrappers
(mpicc, mpicxx, mpif77, mpif90).

I hope this helps,
Gus Correa


Ken Nielson wrote:
> ----- Original Message -----
>> From: "Guilherme Rocha" <guilherme.consultor at gmail.com>
>> To: torqueusers at supercluster.org
>> Sent: Friday, September 2, 2011 3:17:45 AM
>> Subject: [torqueusers] Success setting up a new Torque Environment in	University
>>
>>
>> Hello folks,
>>
>>
>> My name is Guilherme and this is my first post here.
>> Thanks for this great project.
>>
>> We're setting-up a Torque Cluster with 23 nodes and will be used to
>> bioinformatics tasks.
>>
>> I'm completely newbie to all of this, so after hard steps of
>> troubleshooting, we finally
>> received good news in logs. We did a small alginment using clustalw.
>>
>> But we have some doubts about how to use a program in parallel, like:
>>
>>
>> Question 1) I need to have clustalw (or the script programs installed
>> in all nodes?)
>> Clustalw is only installed in head node by now.
>>
>> Question 2: Can we use/open GUI program's interfaces to work using
>> torque?
>>
>> Question 3: When I submit a job, even requesting 10 nodes, clustal
>> are being runned
>> only in one node. What can be wrong?
> When TORQUE launches a mulit-node (parallel) job only one of the execution nodes receives the request from pbs_server. That node is called the mother superior. The mother superior then contacts the other sister nodes for the job and requests the sisters join the job. The script submitted on the qsub command line is then executed on the mother superior. In order to have the job also execute on the sister nodes you need something like MPI or you can write your own program to use the task management API in TORQUE. There is also a utility in TORQUE called pbsdsh that can be used in the job script to make the program requested run on all of the nodes requested in the job.
> 
> Regards
> 
> Ken Nielson
> Adaptive Computing
>>
>> thanks in advance
>>
>> thanks in advance,
>>
>> --
>> --
>> Guilherme Rocha
>> GF7 Doc & Systems - Soluções Tecnológicas
>> Pesquisa e Desenvolvimento - World Wide
>> R. João Goulart, 170 - Rio Pardo - RS - CEP 96640-000
>> Mobile: +55 51 81400360 - Home Page: http://www.gf7.com.br
>


More information about the torqueusers mailing list