[torqueusers] PBS environmental variables and -V

Matthew Britt msbritt at umich.edu
Wed Jan 23 15:30:21 MST 2013

As a counter argument, the value of PBS_NODEFILE isn't getting set to the same value for the second job.   Like PBS_NODEFILE, I would expect the PBS_O variables get set correctly for the submitted job and not the environment passed into it.  It isn't consistent w/ PBS_O variables either, as PBS_O_QUEUE is overwritten w/ the correct value (I exported a different value into PBS_O_QUEUE to check).

fwiw, we're using Environmental Modules, so we set several variables for software packages, like license servers, process launchers (like hydra), etc.  The users might not be aware that these variables are necessary, so we have the users load appropriate module(s) and submit with -V. 

If the PBS_O_ variables are treated differently than PBS_ variables, that's fine; I was curious if it was by design or was a bug.  

 - Matt

Matthew Britt
CAEN HPC Group - College of Engineering
msbritt at umich.edu

On Jan 23, 2013, at 2:54 PM, "Andrus, Brian Contractor" <bdandrus at nps.edu> wrote:

> Seems to me that would be by design and you need to be aware of it.
> -V basically just takes the output of 'env' and sets anything that is set.
> Since you already have PBS_* variables, I would expect them to be set already.
> This could be handled by pbs_mom if it were to first sent the -V stuff and then set the PBS_* stuff.
> You can do this by iterating through and 'unset' all the PBS_* variables before doing the qsub.
> I tend to rend this moot by highly discouraging the use of -V
> It's use makes it difficult to troubleshoot when folks use things like "./a.out" to run their programs.
> Brian Andrus
> ITACS/Research Computing
> Naval Postgraduate School
> Monterey, California
> voice: 831-656-6238
>> -----Original Message-----
>> From: torqueusers-bounces at supercluster.org [mailto:torqueusers-
>> bounces at supercluster.org] On Behalf Of Matthew Britt
>> Sent: Tuesday, January 22, 2013 11:22 AM
>> To: Torque Users Mailing List
>> Subject: [torqueusers] PBS environmental variables and -V
>> Hello everyone.   What is the expected behavior in precedence between
>> PBS environmental variables and -V when a job is submitted  from an
>> interactive job.  With torque 4.1.4 (and possibly earlier), the variables from
>> the environment of the interactive shell are passed along w/ the newly
>> submitted job (either interactive or batch).   We've seen both PBS_O_HOST
>> and PBS_O_WORKDIR be set to values of the first job rather than the
>> attributes of the second job.
>> As an example:
>> [msbritt at nyx ~]$ cd bin
>> [msbritt at nyx bin]$ pwd
>> /home/msbritt/bin
>> [msbritt at nyx bin]$ qsub -I -l nodes=1,walltime=5:00 -q flux -A msbritt_flux -
>> V
>> qsub: waiting for job 9445802.nyx.engin.umich.edu to start
>> qsub: job 9445802.nyx.engin.umich.edu ready
>> [msbritt at nyx5515 ~]$ echo $PBS_O_WORKDIR
>> /home/msbritt/bin
>> [msbritt at nyx5515 ~]$ echo $PBS_O_HOST
>> nyx.engin.umich.edu
>> [msbritt at nyx5515 ~]$ pwd
>> /home/msbritt
>> [msbritt at nyx5515 ~]$ qsub -I -l nodes=1,walltime=5:00 -q flux -A
>> msbritt_flux -V
>> qsub: waiting for job 9445813.nyx.engin.umich.edu to start
>> qsub: job 9445813.nyx.engin.umich.edu ready
>> [msbritt at nyx5623 ~]$ pwd
>> /home/msbritt
>> [msbritt at nyx5623 ~]$ echo $PBS_O_WORKDIR
>> /home/msbritt/bin	(arguably should be /home/msbritt)
>> [msbritt at nyx5623 ~]$ echo $PBS_O_HOST
>> nyx.engin.umich.edu	(arguably should be nyx5515)
>> Should -V not read the PBS_O_* variables on job submission or at least be
>> overwritten and correctly set in the next job, or should -V trump ?
>> Thanks,
>>   - Matt
>> --------------------------------------------
>> Matthew Britt
>> CAEN HPC Group - College of Engineering
>> msbritt at umich.edu
>> _______________________________________________
>> torqueusers mailing list
>> torqueusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/torqueusers
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers

More information about the torqueusers mailing list