[torqueusers] Re: Evaluating DRMAA + Torque
darren at 23andme.com
Wed Jun 4 15:57:18 MDT 2008
> Darren Platt wrote:
>> Thanks! One of the attractions is the portability, so we don't have to
>> recode if we outgrow or change schedulers. Part of the problem seems to be
>> the abstraction. For example, setting the outputPath field in the job to
>> myhost:/some/path/to/some/file.txt didn't deliver the result to the
>> headnode in the way -o would. The problem seemed to be in how the drmaa
>> layer was rewriting the path, assumptions about whether the intermediate
>> path /some/path / etc existed on the node vs the submit host.
> These are the exact sorts of issues I ran into when I was playing with
That's a pity - I assume it's because most users are relying on file systems
for data staging. I managed to get it to at least send results out to the
submit host but only by
patching the source which didn't seem like the intended mode of use ;)
> So I assume most people are using either the shell or torque api to submit
> Absolutely. You might be able to abstract this concept into "job templates"
> that then only need to be modified to fit a new scheduler, allowing you to
> NOT change your code. Generally though, the most common or even popular
> scheduler's accept in the PBS (TORQUE) style submit scripts, so I don't
> think you are taking a risk by using TORQUE.
We are hoping to run things via an API so we can get better integration
between applications and the jobs submitted. Has anyone written a python
API for torque's native API?
Senior Director, Research
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the torqueusers