[torquedev] job arrays?

[Andrew J Caird]

On Tue, 7 Mar 2006, Garrick Staples wrote:

> On Mon, Mar 06, 2006 at 10:15:41PM -0500, Andrew J Caird alleged:
>> On Mon, 6 Mar 2006, Garrick Staples wrote:
>>
>>> On Mon, Mar 06, 2006 at 09:05:14PM -0500, Andrew J Caird alleged:
>>>>
>>>> There was some talk about job arrays appearing in torque at some 
>>>> point, is there any news on this feature?
>>>>
>>>> I doubt I have the skills, and I'm nearly certain I don't have the 
>>>> time, to implement this, but I would be willing to do testing and 
>>>> perhaps some light-weight development if that would be a help.
>>>
>>> Just out of curiosity, have you looked at using mpiexec's scheduling 
>>> features to do basicly the same thing as job arrays?
>>
>> I hadn't, but I will, thanks.
>>
>> That said, are there plans to include this in Torque? :)
>
> I keep thinking about it, but I never seem to come up with a solid idea. 
> I really got stuck when I realized that mpiexec seems to already do it.
>
> Can you outline exactly what you want?  Maybe that will get me moving 
> again.

If I have a job that I want to run with 500 parameters, but I have 100 
computers and 20 other users with limits of 20 nodes per person.  So I 
submit my job array of 500 jobs, and they start when and where they can 
within the constraints of the scheduler - to the scheduler it looks like 
500 jobs.  qsub, qstat, qdel, etc. , though, treat it as one job by 
default, so qdel'ing it kills all of them.  There would be an option to 
qstat to get details out of a job array.

My weak understanding of mpiexec is that it doesn't do this.

Does that make sense?  I am struggling with it myself, so any dialog would 
be appreciated.

--andy