[torquedev] Job array feature request

[Chris Samuel]

On Sat, 17 Nov 2007, Chris Samuel wrote:

> Now as mentioned previously this is just the initial support,
> but it does at least seem to already support single task job
> arrays, but not for an arbitrary array ID.    Non-contiguous
> job arrays are not supported either yet.

Looking at the code in torque-2.2.1/src/server/job_func.c it appears 
that individual tasks are created via the job_clone() function and 
there doesn't appear to be anything in there to stop any of this from 
happening.

But we have a couple of other issues that we need to deal with if we 
want to support this (which I reckon would be a good thing - these 
guys have a genuine use case for this).

Firstly - how do we introduce support for the new syntax ?

Do we change the syntax of the current '-t' option to say that if you 
just give a number then it creates a single task with that index, and 
to create the current form of job array you have to instead do:

qsub -t 0-99 ./pbs-script

or do we leave the -t option as is and introduce a new option, 
say -T ?

Personally I reckon that given the fact that job array support in 
Torque is advertised as "primitive" a user interface change like this 
so early on is worth it just to get things right in the long run.


Secondly, this means we would probably need to store this as a new job 
attribute (a string, the current ATTR_t is a long) and convert older 
style arrays into the new format (not too hard I guess, it just 
becomes "0-($tasks-1)").

Finally the work task function job_clone_wt() would need to support 
the new syntax and intelligently schedule the cloning.

Comments ?

cheers,
Chris
-- 
Christopher Samuel - (03) 9925 4751 - Systems Manager
 The Victorian Partnership for Advanced Computing
 P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url : http://www.supercluster.org/pipermail/torquedev/attachments/20071117/6e0e2b4e/attachment.bin