[torqueusers] Success setting up a new Torque Environment in University

Gus Correa gus at ldeo.columbia.edu
Fri Sep 16 10:48:00 MDT 2011


Bas van der Vlies wrote:
> 
> On 16 sep 2011, at 13:34, Guilherme Rocha wrote:
> 
>>
>>
>>
>> Hello folks,
>>
>>
>>
>> thanks a lot for all answers, I'm doing a troubleshooting now, 
>> considering your answers, in order to get my target acquired.
>> Thanks a lot once more.
>>
>> I already start to research how to use MPI, in fact we have the 
>> clustalw-mpi installed on the head node,
>> but no in the nodes (yet), I will try to do your recommendations now.
>>
>> We alson have open-mpi installed.
>>
>>
>> We are using Debian, all software installed via #aptitude.
>>
>>
>> I have this question from my teacher, the lab coordinator
>>
>>     * Is possible to run X programs (like clustalx)  interactivelly
>>       under Torque?
>>
>>
> qsub - I -X
> 
> -I: interactively 
> -X: X11 forwarding
> 
> You get a shell prompt:
>  * xcalc
> 
> PS) man qsub 
> 

Hi Guilherme and Bas

Guilherme: Here use the 'interctive+X_enabled' feature of Torque
as Bas explained, and it works fine.

However, if your compute nodes are only on a private subnet
[as in most clusters], perhaps doing NAT
to the outer world via head node, a user running an 
interactive+X_enabled job will probably
have to hop from his local machine to the head node (via ssh -X),
and launch the job from there.
This leads to increased network traffic across the head node,
particularly if the program is heavy on graphics.

Here this type of jobs are don't happen very often [to my relief].
Nevertheless, they typically use Matlab, IDL,
this type of multi-windowed
interactive tool that can generate quite a bit of graphic displays, etc,
and large network traffic.
I don't know anything about clustalw, but I would guess it has some
of these charactheristics.

In any case, I presume this is something a system administrator
has to be aware of, and perhaps restrict the number
of simultaneous interactive+X_enabled jobs, say, via a specific queue,
to avoid a bottleneck.

Other people on the list with more experience with interactive+X_enabled
jobs could perhaps post their views.

Thank you,
Gus Correa


>>
>>
>>
>>
>> 2011/9/2 Coyle, James J [ITACD] <jjc at iastate.edu <mailto:jjc at iastate.edu>>
>>
>>       You’ll need and MPI application to use multiple nodes.____
>>
>>     __ __
>>
>>       Perhaps this application would be  Clustalw-MPI .____
>>
>>     It looks like this is available at:____
>>
>>     http://www.bii.a-star.edu.sg/achievements/applications/clustalw/download.php____
>>
>>     Information on this can be found at:____
>>
>>     http://www.bii.a-star.edu.sg/docs/software/README.clustalw-mpi____
>>
>>     __ __
>>
>>     __ __
>>
>>       Torque just reserves node, you need something like MPi and
>>     program written in MPI to use____
>>
>>     multiple nodes.____
>>
>>     __ __
>>
>>       If you need a suggestion on MPI,  I use OpenMPI because it
>>     installs easily and can use ____
>>
>>     multiple network interconnects, it also works well with Torque.____
>>
>>     __ __
>>
>>     Ethernet works, but is slow, however, that is likely what you
>>     have.____
>>
>>     We use Infiniband and Myrinet networks in addition to Ethernet.
>>     They give much better ____
>>
>>     performance for our workloads, but the cards and switches are very
>>     expensive.____
>>
>>     __ __
>>
>>     __ __
>>
>>     *From:* torqueusers-bounces at supercluster.org
>>     <mailto:torqueusers-bounces at supercluster.org>
>>     [mailto:torqueusers-bounces at supercluster.org
>>     <mailto:torqueusers-bounces at supercluster.org>] *On Behalf Of
>>     *Guilherme Rocha
>>     *Sent:* Friday, September 02, 2011 4:18 AM
>>     *To:* torqueusers at supercluster.org
>>     <mailto:torqueusers at supercluster.org>
>>
>>
>>     *Subject:* [torqueusers] Success setting up a new Torque
>>     Environment in University____
>>
>>     __ __
>>
>>     Hello folks,
>>
>>
>>
>>
>>     My name is Guilherme and this is my first post here.
>>     Thanks for this great project.
>>
>>     We're setting-up a Torque Cluster with 23 nodes and will be used
>>     to bioinformatics tasks.
>>
>>     I'm completely newbie to all of this, so after hard steps of
>>     troubleshooting, we finally
>>     received good news in logs.  We did a small alginment using clustalw.
>>
>>     But we have some doubts about how to use a program in parallel, like:
>>
>>
>>     Question 1)  I need to have clustalw (or the script programs
>>     installed in all nodes?)
>>     Clustalw is only installed  in head node by now.
>>
>>     Question 2:  Can we use/open GUI program's interfaces to work 
>>     using torque?
>>
>>     Question 3: When I submit a job, even requesting 10 nodes, clustal
>>     are being runned
>>     only in one node. What can be wrong?
>>
>>
>>     thanks in advance
>>
>>     thanks in advance,
>>
>>
>>     -- 
>>     -- 
>>     Guilherme Rocha
>>     GF7 Doc & Systems - Soluções Tecnológicas
>>     Pesquisa e Desenvolvimento - World Wide
>>     R. João Goulart, 170  - Rio Pardo - RS - CEP 96640-000
>>     Mobile:  +55 51 81400360 <tel:%2B55%2051%2081400360> - Home
>>     Page:   http://www.gf7.com.br <http://www.gf7.com.br/> ____
>>
>>
>>     _______________________________________________
>>     torqueusers mailing list
>>     torqueusers at supercluster.org <mailto:torqueusers at supercluster.org>
>>     http://www.supercluster.org/mailman/listinfo/torqueusers
>>
>>
>>
>>
>> -- 
>> -- 
>> Guilherme Rocha
>> GF7 Doc & Systems - Soluções Tecnológicas
>> Pesquisa e Desenvolvimento - World Wide
>> R. João Goulart, 170  - Rio Pardo - RS - CEP 96640-000
>> Mobile:  +55 51 81400360 - Home Page:   http://www.gf7.com.br 
>> <http://www.gf7.com.br/>
>>
>> <ATT00001..txt>
> 
> --
> Bas van der Vlies
> basv at sara.nl <mailto:basv at sara.nl>
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers



More information about the torqueusers mailing list