[torqueusers] How to run mpirun of intel on torque

Coyle, James J [ITACD] jjc at iastate.edu
Fri Dec 14 13:05:24 MST 2012


   I assume this response for some other message.

-          Jim C.

From: torqueusers-bounces at supercluster.org [mailto:torqueusers-bounces at supercluster.org] On Behalf Of Brian Haymore
Sent: Friday, December 14, 2012 12:56 PM
To: Torque Users Mailing List
Subject: Re: [torqueusers] How to run mpirun of intel on torque

There are no quotas present on the scratch file systems. So for you to be getting a quota limited  message tells me that somehow the tool you're using is trying to write something to your user home directory. You should look  to clean up some of your home directory to bring yourself below your quota limit .


Brian D Haymore

University of Utah

Center for High Performance Computing

155 South 1452 East, RM 405

Salt Lake City, Utah 84112

Phone: 801-558-1150, Fax: 801-585-5366


"Coyle, James J [ITACD]" <jjc at iastate.edu<mailto:jjc at iastate.edu>> wrote:

  I have a script that I run from /var/spool/mom_priv/prologue whenever a josb that
requires all the cores on the node.  This script kills any user level processes that are not
the job owner or a torque manager.

  This gets rid of any effect of users who should not be logged on to the compute node, that
is who have just ssh'd in rather than using qsub -I

   I clean out scratch space on exit with epilogue if it is a dedicated job.

    You could couple this with techniques to prevent ssh logins durin the job
(For jobs which dedicate the node, have prologue disallow logins from other users,
and restore normal function on exit.)

From: torqueusers-bounces at supercluster.org<mailto:torqueusers-bounces at supercluster.org> [mailto:torqueusers-bounces at supercluster.org] On Behalf Of David Roman
Sent: Friday, December 14, 2012 6:53 AM
To: 'torqueusers at supercluster.org'
Subject: [torqueusers] How to run mpirun of intel on torque


I am sorry, but my english is really sad. I thank you for your patience for that.
I installed TORQUE 4.1.0, with MAU 3.3.1. I compiled OPENMPI with option --with-tm=/usr/local/torque.
I disallowed ssh connections for my users on executable nodes. In /etc/ssh/sshd_config I set
AllowGroups root admin
This works fine.

But now, I install MPI Intel Librarie and Ifort.
When I open a interactive pbs session:
qsub -I -l nodes=2:ppn=8

I am connected on a node.

I run mpi job

mpirun -genv I_MPI_FABRICS_LIST tmi ./my_program

But it can  not start, because it cannot connect on the other node.

If I append users in AllowGroups  of  /etc/ssh/sshd_config it works.

But if I do this, all users can connect on executable nodes, without use torque, and this is bad.

How can I do to disallow ssh connection without torque or make mpirun of intel works like openmpi, without ssh connection allowed for users ?

Thank you


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20121214/1179c71d/attachment-0001.html 

More information about the torqueusers mailing list