[torqueusers] a bit o' help with torque paths and such

Ricardo Román Brenes rroman at cenat.ac.cr
Mon Aug 6 17:00:48 MDT 2012


Hello guys

I got a problem here, ill be as direct as i can be.

I got 4 nodes, on node 0 there's torque/maui server and a mom, and on the
1-3 theres just the torque mom.

my problem is that when i do "exports" as a user using a ssh login i get
paths taht i dont get when i launch either an interactive job or a batch
job using qsub:
As a regular ssh login with a non-root user:

> [rroman at zarate-0 ~]$ export
>
>
> declare -x BASHERRECE="bashrc"
>
>
> declare -x BASHLOCAL=".bashrc"
>
>
> declare -x G_BROKEN_FILENAMES="1"
>
>
> declare -x HISTSIZE="1000"
>
>
> declare -x HOME="/home/rroman"
>
>
> declare -x HOSTNAME="zarate-0.cnca"
>
>
> declare -x KDEDIRS="/usr"
>
>
> declare -x KDE_IS_PRELINKED="1"
>
>
> declare -x KMIX_PULSEAUDIO_DISABLE="1"
>
>
> declare -x LANG="en_US.UTF-8"
>
>
> declare -x LD_LIBRARY_PATH="/usr/local/lib/:"
>
>
> declare -x LESSOPEN="|/usr/bin/lesspipe.sh %s"
>
>
> declare -x LOGNAME="rroman"
>
>
> declare -x
> LS_COLORS="rs=0:di=01;34:ln=01;36:hl=44;37:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;4
>
> 2:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz
>
> =01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.
>
> rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01
>
> ;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*
>
> .nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;3
>
> 5:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra
> =01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:"
>
>
> declare -x MAIL="/var/spool/mail/rroman"
>
>
> declare -x OLDPWD
>
>
> declare -x
> PATH="/usr/lib/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/maui/bin:/usr/local/maui/sbin
> :/usr/local/sbin:/usr/local/bin:/home/rroman/bin"
>
>
> declare -x PERFIL="profile"
>
>
> declare -x PWD="/home/rroman"
>
>
> declare -x QTDIR="/usr/lib/qt-3.3"
>
>
> declare -x QTINC="/usr/lib/qt-3.3/include"
>
>
> declare -x QTLIB="/usr/lib/qt-3.3/lib"
>
>
> declare -x SHELL="/bin/bash"
>
>
> declare -x SHLVL="1"
>
>
> declare -x SSH_CLIENT="192.168.1.50 64228 22"
>
>
> declare -x SSH_CONNECTION="192.168.1.50 64228 192.168.1.200 22"
>
>
> declare -x SSH_TTY="/dev/pts/1"
>
>
> declare -x TERM="cygwin"
>
>
> declare -x TZ="America/Costa_Rica"
>
>
> declare -x USER="rroman"
>
>


Please note the BASHERRECE, BASHLOCAL and PERFIL variables. these are
defined in /etc/bashrc, /home/$USER/.bashrc and /etc/profile, also check
the PATH, it has /usr/local/bin

now with a interactive job (or batch, the results are the same) using qsub:

> [rroman at zarate-0 jobs]$ qsub -q batch ./a.sh
>
>
> 25.zarate-0.cnca
>
>
> [rroman at zarate-0 jobs]$ cat a.sh.o
>
>
> a.sh.o12  a.sh.o14  a.sh.o15  a.sh.o21  a.sh.o25
>
>
> [rroman at zarate-0 jobs]$ cat a.sh.o25
>
>
> uid=500(rroman) gid=503(cluster) groups=503(cluster)
>
>
> Mon Aug  6 16:49:44 CST 2012
>
>
> zarate-3.cnca
>
>
> /usr/lib/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/bin:/usr/bin:/local/sbin:/usr/local/sbin:/usr/sbin:/sbin:/home/rroman/bin
>
> declare -x BASHLOCAL=".bashrc"
>
>
> declare -x ENVIRONMENT="BATCH"
>
>
> declare -x G_BROKEN_FILENAMES="1"
>
>
> declare -x HISTSIZE="1000"
>
>
> declare -x HOME="/home/rroman"
>
>
> declare -x HOSTNAME="zarate-3.cnca"
>
>
> declare -x KDEDIRS="/usr"
>
>
> declare -x KDE_IS_PRELINKED="1"
>
>
> declare -x KMIX_PULSEAUDIO_DISABLE="1"
>
>
> declare -x LANG="C"
>
>
> declare -x LESSOPEN="|/usr/bin/lesspipe.sh %s"
>
>
> declare -x LOGNAME="rroman"
>
>
> declare -x MAIL="/var/spool/mail/rroman"
>
>
> declare -x OLDPWD
>
>
> declare -x
> PATH="/usr/lib/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/bin:/usr/bin:/local/sbin:/usr/local/sbin:/usr/sbin:/sbin:/home/rroman/bin"
>
> declare -x PBS_ENVIRONMENT="PBS_BATCH"
>
>
> declare -x PBS_JOBCOOKIE="D6AD4C58275CD48820987209370F61CA"
>
>
> declare -x PBS_JOBID="25.zarate-0.cnca"
>
>
> declare -x PBS_JOBNAME="a.sh"
>
>
> declare -x PBS_MOMPORT="15003"
>
>
> declare -x PBS_NODEFILE="/var/torque/aux//25.zarate-0.cnca"
>
>
> declare -x PBS_NODENUM="0"
>
>
> declare -x PBS_O_HOME="/home/rroman"
>
>
> declare -x PBS_O_HOST="zarate-0.cnca"
>
>
> declare -x PBS_O_LANG="en_US.UTF-8"
>
>
> declare -x PBS_O_LOGNAME="rroman"
>
>
> declare -x PBS_O_MAIL="/var/spool/mail/rroman"
>
>
> declare -x
> PBS_O_PATH="/usr/lib/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/maui/bin:/usr/local/maui/sbin:/usr/local/sbin:/usr/local/bin:/home/rroman/bin"
>
>
> declare -x PBS_O_QUEUE="batch"
>
>
> declare -x PBS_O_SHELL="/bin/bash"
>
>
> declare -x PBS_O_TZ="America/Costa_Rica"
>
>
> declare -x PBS_O_WORKDIR="/home/rroman/jobs"
>
>
> declare -x PBS_QUEUE="batch"
>
>
> declare -x PBS_TASKNUM="1"
>
>
> declare -x PBS_VNODENUM="0"
>
>
> declare -x PWD="/home/rroman"
>
>
> declare -x QTDIR="/usr/lib/qt-3.3"
>
>
> declare -x QTINC="/usr/lib/qt-3.3/include"
>
>
> declare -x QTLIB="/usr/lib/qt-3.3/lib"
>
>
> declare -x SHELL="/bin/bash"
>
>
> declare -x SHLVL="2"
>
>
> declare -x USER="rroman"
>
>
> [rroman at zarate-0 jobs]$
>
>


See that there is no BASHERRECE or PERFIL. And the paths are different.

My problem is taht the mpicc is in /usr/local/bin and i need it in the pbs
jobs, and i would really like NOT to add that path in the pbs jobs scripts,
so i need to make it default but i dont know why it doenst work...
Check both paths in a normal login and with an interactive job:

> [rroman at zarate-0 jobs]$ echo $PATH
>
>
> /usr/lib/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/maui/bin:/usr/local/maui/sbin:/usr/local/sbin:/usr/local/bin:/home/rroman/bin
>
>
> [rroman at zarate-0 jobs]$ qsub -I -q batch
>
>
> qsub: waiting for job 27.zarate-0.cnca to start
>
>
> eqsub: job 27.zarate-0.cnca ready
>
>
>
>
>
> [rroman at zarate-3 ~]$ echo $PATH
>
>
> /usr/lib/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/bin:/usr/bin:/local/sbin:/usr/local/sbin:/usr/sbin:/sbin:/home/rroman/bin
>



Im using centos 6, torque 2.1.10 and mayu 3.3.1

SO FINALLY THE QUESTIONS:

1. Which scripts are run when i launch a PBS job? (/etc/bashrc?
/etc/profile.d/*?)
2. how can i add this /usr/local/bin to the path of all pbs jobs?

thanks. If more info is needed please point it out! :)

-ricardo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20120806/60dec571/attachment-0001.html 


More information about the torqueusers mailing list