[torqueusers] Step Change in Job Arrays

Sreedhar Manchu sm4082 at nyu.edu
Wed Feb 29 20:51:03 MST 2012


Hello,

Putting the code below in qsub wrapper would make torque recognize step
change like 1-20:2 or 0-19:3, etc. This works only when -t is mentioned in
the command line. I made my script recognize the same even when users
mention it inside the script as well. But it's quite long. I guess once we
find #PBS -t 2-23:3, we just have to make the script change it to #PBS -t
2,5,8,11,14,17,20,23. Which can be easily done with Andy's method of using
seq.

Anyway, here is the code for wrapper for making it work with command line.

#!/bin/sh
args=("$@")
for((arg=0;arg<$#;arg++))
do
    if [ "${args[$arg]}" = "-t" ]
    then
        if echo "${args[$(($arg+1))]}" | egrep
"^(^0|^[1-9][0-9]*)-[1-9][0-9]*:[1-9][0-9]*$" > /dev/null 2>&1
        then
            array_arg="${args[$(($arg+1))]}"
            array_arg="$(seq -s, $(echo $array_arg | cut -f1 -d-) $(echo
$array_arg | cut -f2 -d:) $(echo $array_arg | cut -f1 -d: | cut -f2 -d-))"
            str="set -- \"\${@:1:$(($arg+1))}\" \"${array_arg}\"
\"\${@:$(($arg+3))}\""
            eval `echo "$str"`
            break
        fi
    fi
done

In case you already have a wrapper in place, it doesn't do any harm to keep
it at the beginning of it.

Sreedhar.


On Wed, Feb 8, 2012 at 9:47 AM, Ibad Kureshi U0850037 <U0850037 at hud.ac.uk>wrote:

> Thanks Glen, Andy
>
> Andy: Nice!
>
> -Ibad
>
>
> ________________________________________
> From: torqueusers-bounces at supercluster.org [
> torqueusers-bounces at supercluster.org] On Behalf Of Andrew Caird [
> acaird at umich.edu]
> Sent: Wednesday, February 08, 2012 2:28 PM
> To: Torque Users Mailing List
> Subject: Re: [torqueusers] Step Change in Job Arrays
>
> On Wed, Feb 8, 2012 at 9:02 AM, Glen Beane <glen.beane at gmail.com<mailto:
> glen.beane at gmail.com>> wrote:
> On Wed, Feb 8, 2012 at 8:57 AM, Ibad Kureshi U0850037
> <U0850037 at hud.ac.uk<mailto:U0850037 at hud.ac.uk>> wrote:
> > Hello,
> >
> > I was wondering is someone could tell me how to adjust the step size in
> a job array. We are running Torque 2.5.7 with the PBS_SCHEDD on a small
> cluster and our users want to submit arrays.
> >
> > One the SGE and the Moab/Torque based systems
> >
> > $ -t 1-20:2
> >
> > or
> >
> > #PBS -t 1-20:2
> >
> > respectively, gives them 10 jobs with even ID numbers.
> >
> > How can this be done with Torque? It throws out "qsub: Bad Job Array
> Request" error
> >
> > Have not been able to find much literature on this.
> >
> > Thanks
>
>
> this is not currently supported, but it is a great feature request.
>
> unfortunately the only option would be to explicitly specify each array ID:
>
> #PBS -t 2,4,6,8,10 ...20
>
> Or:
>
>  qsub -t `seq -s, 2 2 20` pbsfile.txt
>
> in case you don't want to type all the numbers.
>
> --andy
>
>
> ---
> This transmission is confidential and may be legally privileged. If you
> receive it in error, please notify us immediately by e-mail and remove it
> from your system. If the content of this e-mail does not relate to the
> business of the University of Huddersfield, then we do not endorse it and
> will accept no liability.
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers
>



-- 
Sreedhar Manchu
HPC Support Specialist
New York University
251 Mercer Street
New York, NY 10012-1110
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20120229/541dacf5/attachment.html 


More information about the torqueusers mailing list