[torquedev] Job array feature request

Glen Beane glen.beane at gmail.com
Mon Nov 19 06:11:21 MST 2007


On 11/16/07, Chris Samuel <csamuel at vpac.org> wrote:
> On Sat, 17 Nov 2007, Chris Samuel wrote:
>
> > Now as mentioned previously this is just the initial support,
> > but it does at least seem to already support single task job
> > arrays, but not for an arbitrary array ID. Non-contiguous
> > job arrays are not supported either yet.
>
> Looking at the code in torque-2.2.1/src/server/job_func.c it appears
> that individual tasks are created via the job_clone() function and
> there doesn't appear to be anything in there to stop any of this from
> happening.
>
> But we have a couple of other issues that we need to deal with if we
> want to support this (which I reckon would be a good thing - these
> guys have a genuine use case for this).
>
> Firstly - how do we introduce support for the new syntax ?
>
> Do we change the syntax of the current '-t' option to say that if you
> just give a number then it creates a single task with that index, and
> to create the current form of job array you have to instead do:
>
> qsub -t 0-99 ./pbs-script
>
> or do we leave the -t option as is and introduce a new option,
> say -T ?
>
> Personally I reckon that given the fact that job array support in
> Torque is advertised as "primitive" a user interface change like this
> so early on is worth it just to get things right in the long run.
>
>
> Secondly, this means we would probably need to store this as a new job
> attribute (a string, the current ATTR_t is a long) and convert older
> style arrays into the new format (not too hard I guess, it just
> becomes "0-($tasks-1)").
>
> Finally the work task function job_clone_wt() would need to support
> the new syntax and intelligently schedule the cloning.
>
> Comments ?

I have no problem changing the syntax


More information about the torquedev mailing list