[torqueusers] still having modules environment / Torque shell interaction problems

David Backeberg david.backeberg at case.edu
Sat Jun 9 14:04:47 MDT 2007


I have a working solution now, I modified the script packed with
modules-environment 3.2.5, and added one line, just below where the
function modules is defined.

export -f modules

It occurred to me that all the environmental variables were making it
into the PBS shell, and maybe I needed to find a way to export the
function too. With the export -f in place, I'm now getting modules
working within the PBS shell, without having to do the
bash -l
trick.

Thanks for everybody helping me with this,
Dave

below is what's currently in my /etc/profile.d/modules.sh, and working
to get modules into my PBS shell...
if [ "${MODULE_VERSION:-}" = "" ]; then
        MODULE_VERSION_STACK="3.2.5"
        MODULE_VERSION="3.2.5"
        export MODULE_VERSION
else
        MODULE_VERSION_STACK="$MODULE_VERSION"
fi
export MODULE_VERSION_STACK

module() { eval
`/usr/local/modules-env-3.2.5/Modules/$MODULE_VERSION/bin/modulecmd
bash $*`; }
#module() { eval
`/usr/local/modules-env-3.2.5/Modules/3.2.5/bin/modulecmd bash $*`; }
export -f module

MODULESHOME=/usr/local/modules-env-3.2.5/Modules/3.2.5
export MODULESHOME

if [ "${LOADEDMODULES:-}" = "" ]; then
  LOADEDMODULES=
  export LOADEDMODULES
fi

if [ "${MODULEPATH:-}" = "" ]; then
  MODULEPATH=`sed 's/#.*$//' ${MODULESHOME}/init/.modulespath | awk
'NF==1{printf("%s:",$1)}'`
  export MODULEPATH
fi


On 6/8/07, Michel Béland <michel.beland at rqchp.qc.ca> wrote:
> Hello David,
>
> > I jumped the gun this afternoon in saying the problem has been
> > resolved with my Torque / modules environment interaction. (in the
> > thread with title about prologue / epilogue).
>
> We are using modules (version 3.1.6) here on SGI Altix machines and on a
> Cray XD1. We are using PBS Pro instead of Torque, but I guess the
> solutions to your problems will be similar.
>
> > Here's what I've been able to determine by adding comments to various
> > /etc/profile and /etc/profile.d scripts.
> >
> > 1) Torque definitely munges /etc/profile, and then /etc/profile.d/*.sh
> > scripts when setting up the Torque shell.
>
> Yes, it reads these files. Here, we have PBS working in a way that PBS
> first starts a login shell (either the users's default one or the shell
> specified with -S), then starts the user's script in whatever shell it
> happens to be written in, determined from the first line (#!/bin/ksh,
> for example). So the login shell reads /etc/profile and
> /etc/profile.d/*.sh. But when the user's script is run, it does not read
> them again. The environment is passed to the user's script, but the
> shell functions are not.
>
> One solution might be to export the functions, but as we tried to make
> this work for all login shells (csh, bash, tcsh, zsh, sh, ksh) this was
> not possible. We tried to define ENV to point to a file which would
> define the module function, but it worked with pdksh, but not with the
> real ksh (if I recall correctly). For the real ksh, you define the
> environment variable FPATH to point to a directory where you have a file
> called module in which you define the function module.
>
> > 2) variables set in the /etc/profile.d/modules-env.sh and export-ed,
> > are properly set in the PBS script environment. For instance, I can
> > echo $MODULEPATH and the correct value is displayed.
> >
> > 3) for reasons I haven't been able to determine, the actual modules
> > command somehow escapes the PBS script environment, so I'm still
> > getting "command not found" when using the module command in my PBS
> > scripts, but it's not because /etc/profile.d/modules.sh. It's as if
> > the PBS shell somehow gets amnesia, or forks off a clone shell that
> > doesn't get the value.
>
> The explanation above should make it clear.
>
> Here, we manage to make the module command work for all combinations of
> the most current shells (sh, (pd)ksh, bash, csh, tcsh and zsh), for the
> login shells and the shell used in the script itself. It got rather
> involved. It is pretty easy for csh and tcsh as they read
> /etc/csh.cshrc, at least on our system. It was also easy for zsh. But
> for bash and pdksh, we had to make sure that environment variables ENV
> and BASH_ENV were defined in all shells and would point to a file
> containing the module function definition. For sh, it places functions
> definitions on the environment, so we put it there first and it worked.
> On way to do it with PBS Pro is to put it the module function definition
> in $PBS_HOME/pbs_environment. I suppose that this file exists for Torque
> too. Here is what this file looks like at our site on our Altix :
>
> PATH=/bin:/usr/bin
> MPI_DSM_DISTRIBUTE=1
> module=() { eval `/usr/bin/modulecmd sh $*`; }
>
> Hope this helps,
>
> --
> Michel Béland, analyste en calcul scientifique
> michel.beland at rqchp.qc.ca
> bureau S-222, pavillon Roger-Gaudry (principal), Université de Montréal
> téléphone   : (514) 343-6111 poste 3892     télécopieur : (514) 343-2155
> RQCHP (Réseau québécois de calcul de haute performance)  www.rqchp.qc.ca
>


More information about the torqueusers mailing list