[torqueusers] mpi libraries not being loaded with torque

[Garrick Staples]

On Mon, Sep 10, 2007 at 04:25:07PM -0500, Adams, Samuel D Contr AFRL/HEDR alleged:
> I am trying to make my new cluster flexible such that it can run with
> more than one configuration at the same time.  For example, you can
> choose gcc, pg, or Intel compilers using OpenMPI.  To start out with, I
> am just using gcc 4.1 that comes with RHEL5 and OpenMPI.  For some
> reason, I am having trouble with the way it is loading the libraries
> depending on how I run the job.  Basically it would seem that the
> LD_LIBRARY_PATH is not set properly depending one how I run the job; it
> works interactively but not with torque.  
> 
> I have this set in my .bashrc file in the root of my home directory
> 
> if [ `hostname | grep "prod"` ]; then
>         PATH=/usr/local/profiles/gcc-openmpi/bin/:$PATH
>  
> LD_LIBRARY_PATH=/usr/local/profiles/gcc-openmpi/lib/:$LD_LIBRARY_PATH
> fi

That would be incorrect.  bash doesn't read .bashrc on non-interactive login
shells.  See the INVOCATION section of the bash manpage.

Personally, I'd make sure the openmpi binaries were built with the correct
rpaths and refuse to LD_LIBRARY_PATH.  Then either call mpirun with the full
path or set PATH at the top of the batch script.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/torqueusers/attachments/20070910/5b48f6cf/attachment.bin