[torqueusers] PBS environmental variables and -V

Andrus, Brian Contractor bdandrus at nps.edu
Fri Jan 25 19:37:14 MST 2013


We use environment modules too. I have folks put those in their scripts.

#PBS -l procs=10
module compile/pgi mpi/openmpi
mpirun myprog

Now when they say my program is not running, I can easily replicate their environment because it is all in the script AND they don't shoot themselves in the foot if they are submitting a job from within a job.

What could work if you are using -V is something like:

for val in $(env|awk -F= '/^PBS/{print $1}'); do
echo $val=${!val}>/tmp/$PBS_JOBID.sh;
unset $val;
done
qsub -V somescript.sh
source /tmp/$PBS_JOBID.sh
rm /tmp/$PBS_JOBID.sh


Brian Andrus
ITACS/Research Computing
Naval Postgraduate School
Monterey, California
voice: 831-656-6238




From: torqueusers-bounces at supercluster.org [mailto:torqueusers-bounces at supercluster.org] On Behalf Of Ken Nielson
Sent: Wednesday, January 23, 2013 3:11 PM
To: Torque Users Mailing List
Subject: Re: [torqueusers] PBS environmental variables and -V

Matt,

I guess you would call it a bug. I think it is simply years of maintenance without knowledge of the original intent. Rick made a ticket for this and we will see what it currently does and try to figure out what it should do and then let everyone know.

Ken
On Wed, Jan 23, 2013 at 3:30 PM, Matthew Britt <msbritt at umich.edu<mailto:msbritt at umich.edu>> wrote:
As a counter argument, the value of PBS_NODEFILE isn't getting set to the same value for the second job.   Like PBS_NODEFILE, I would expect the PBS_O variables get set correctly for the submitted job and not the environment passed into it.  It isn't consistent w/ PBS_O variables either, as PBS_O_QUEUE is overwritten w/ the correct value (I exported a different value into PBS_O_QUEUE to check).

fwiw, we're using Environmental Modules, so we set several variables for software packages, like license servers, process launchers (like hydra), etc.  The users might not be aware that these variables are necessary, so we have the users load appropriate module(s) and submit with -V.

If the PBS_O_ variables are treated differently than PBS_ variables, that's fine; I was curious if it was by design or was a bug.

 - Matt

--------------------------------------------
Matthew Britt
CAEN HPC Group - College of Engineering
msbritt at umich.edu<mailto:msbritt at umich.edu>

On Jan 23, 2013, at 2:54 PM, "Andrus, Brian Contractor" <bdandrus at nps.edu<mailto:bdandrus at nps.edu>> wrote:

> Seems to me that would be by design and you need to be aware of it.
> -V basically just takes the output of 'env' and sets anything that is set.
> Since you already have PBS_* variables, I would expect them to be set already.
>
> This could be handled by pbs_mom if it were to first sent the -V stuff and then set the PBS_* stuff.
> You can do this by iterating through and 'unset' all the PBS_* variables before doing the qsub.
>
> I tend to rend this moot by highly discouraging the use of -V
> It's use makes it difficult to troubleshoot when folks use things like "./a.out" to run their programs.
>
>
> Brian Andrus
> ITACS/Research Computing
> Naval Postgraduate School
> Monterey, California
> voice: 831-656-6238<tel:831-656-6238>
>
>
>
>> -----Original Message-----
>> From: torqueusers-bounces at supercluster.org<mailto:torqueusers-bounces at supercluster.org> [mailto:torqueusers-<mailto:torqueusers->
>> bounces at supercluster.org<mailto:bounces at supercluster.org>] On Behalf Of Matthew Britt
>> Sent: Tuesday, January 22, 2013 11:22 AM
>> To: Torque Users Mailing List
>> Subject: [torqueusers] PBS environmental variables and -V
>>
>> Hello everyone.   What is the expected behavior in precedence between
>> PBS environmental variables and -V when a job is submitted  from an
>> interactive job.  With torque 4.1.4 (and possibly earlier), the variables from
>> the environment of the interactive shell are passed along w/ the newly
>> submitted job (either interactive or batch).   We've seen both PBS_O_HOST
>> and PBS_O_WORKDIR be set to values of the first job rather than the
>> attributes of the second job.
>>
>> As an example:
>>
>> [msbritt at nyx ~]$ cd bin
>> [msbritt at nyx bin]$ pwd
>> /home/msbritt/bin
>> [msbritt at nyx bin]$ qsub -I -l nodes=1,walltime=5:00 -q flux -A msbritt_flux -
>> V
>> qsub: waiting for job 9445802.nyx.engin.umich.edu<http://9445802.nyx.engin.umich.edu> to start
>> qsub: job 9445802.nyx.engin.umich.edu<http://9445802.nyx.engin.umich.edu> ready
>>
>> [msbritt at nyx5515 ~]$ echo $PBS_O_WORKDIR
>> /home/msbritt/bin
>> [msbritt at nyx5515 ~]$ echo $PBS_O_HOST
>> nyx.engin.umich.edu<http://nyx.engin.umich.edu>
>> [msbritt at nyx5515 ~]$ pwd
>> /home/msbritt
>>
>> [msbritt at nyx5515 ~]$ qsub -I -l nodes=1,walltime=5:00 -q flux -A
>> msbritt_flux -V
>> qsub: waiting for job 9445813.nyx.engin.umich.edu<http://9445813.nyx.engin.umich.edu> to start
>> qsub: job 9445813.nyx.engin.umich.edu<http://9445813.nyx.engin.umich.edu> ready
>>
>> [msbritt at nyx5623 ~]$ pwd
>> /home/msbritt
>> [msbritt at nyx5623 ~]$ echo $PBS_O_WORKDIR
>> /home/msbritt/bin    (arguably should be /home/msbritt)
>> [msbritt at nyx5623 ~]$ echo $PBS_O_HOST
>> nyx.engin.umich.edu<http://nyx.engin.umich.edu>  (arguably should be nyx5515)
>>
>>
>> Should -V not read the PBS_O_* variables on job submission or at least be
>> overwritten and correctly set in the next job, or should -V trump ?
>>
>> Thanks,
>>   - Matt
>>
>> --------------------------------------------
>> Matthew Britt
>> CAEN HPC Group - College of Engineering
>> msbritt at umich.edu<mailto:msbritt at umich.edu>
>>
>>
>> _______________________________________________
>> torqueusers mailing list
>> torqueusers at supercluster.org<mailto:torqueusers at supercluster.org>
>> http://www.supercluster.org/mailman/listinfo/torqueusers
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org<mailto:torqueusers at supercluster.org>
> http://www.supercluster.org/mailman/listinfo/torqueusers

_______________________________________________
torqueusers mailing list
torqueusers at supercluster.org<mailto:torqueusers at supercluster.org>
http://www.supercluster.org/mailman/listinfo/torqueusers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20130126/7571dc27/attachment-0001.html 


More information about the torqueusers mailing list