[torqueusers] qsub environment variable limits

[Dave Jackson]

Garrick,

  Do you already know which variable is overflowing?  Do you have a gdb
'where' output at a failure point?  We would like to, at a minimum,
either make the variable dynamic or at least bounds check it and report
a clean failure (as well as document the limitation)

Dave

On Mon, 2005-07-11 at 16:37 -0700, Garrick Staples wrote:
> On Mon, Jul 11, 2005 at 11:48:31AM -0700, Josh Lauricha alleged:
> > Specifically, what are they? I have a user doing something along the
> > lines of:
> > 
> >  $ find -type f -maxdepth 1 | xargs -n 250 echo | while read INPUT; do
> >         export INPUT; qsub foo.sh; done
> > 
> > with foo.sh something like:
> > #!/bin/sh
> > #PBS -v INPUT
> > 
> > for file in $INPUT; do
> >     something_to $file
> > done
> > 
> > after around 2-4k of data, the buffer used to send the data overflows.
> > It might be helpful for an error message to be printed when they are
> > going to be truncated.
> 
> It is definitely not checking bounds, in fact I'm getting segfaults in qsbu
> trying to set insanely large variables.  Feel free to file a bug in
> http://clusterresources/bugzilla
> 
> But in all honestly, you are fighting an uphill battle here with really large
> env vars.  You'd be better off putting the list of input files into the job
> script.
> 
> find -type f -maxdepth 1 | xargs -n 250 echo | \
>    while read INPUT; do
>       sed "s|%INPUT%|$INPUT|" < foo.sh | qsub -N foo.sh
>    done
> 
> 
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers