[torquedev] java native interface
mamonski at man.poznan.pl
Wed Oct 20 03:58:08 MDT 2010
On 20 October 2010 11:41, Scott Wilson <scott at epistemy.com> wrote:
> Actually, 1.0.2 had everything I needed, it's just that a few things I didn't need but threw in by habit (like -r n) weren't there, and it could come as a nasty surprise to anyone used to using them. Even then there was a nice error message explaining what was wrong. Glad to see that development is continuing - 1.0.3 has re-instated all the options I'd ever used.
I'm happy to hear that. I general i believe that are many advantages
of using DRMAA over spawning qsub and qstat processes. First is
reduced cost of migrating between different queuing systems (you need
change only how do you construct the native specification attribute -
which is usually needed only for more complex scenarios). The second
asset is that it should be more efficient (no need to fork a new
process, the 1.0.3 version can be also configured to use torque log
files to get job status change events instead of polling).
I have also question to the Torque dev people. Is there a chance to
merge a new version of DRMAA into the trunk?
> Scott Wilson
> On 19 Oct 2010, at 12:32, Mariusz Mamoński wrote:
>> I have just released a newer version. It is available at sourceforge:
>> It implements much more option in the native specification (please let
>> us now which one are still missing in your opinion).
>> On 19 October 2010 11:07, Scott Wilson <scott at epistemy.com> wrote:
>>> In short: no.
>>> How I've been accessing TORQUE is through Dan Templeton's Java DRMAA bindings for Sun/Oracle Grid Engine. He was good enough to make them a fairly thin wrapping round the C DRMAA SGE bindings, so they're fairly straight forwards to port to use another C DRMAA binding (see http://blogs.sun.com/templedf/entry/porting_the_drmaa_java_language ). I've been using the FedStage ones from http://fury.man.poznan.pl/~mmamonski/PL-Grid/pbs_drmaa-1.0.2.tar.gz as the ones bundled with TORQUE are out of date, don't support half the methods and, as you have no doubt been discovering, are pretty much un-documented.
>>> Even those aren't perfect. Thanks to DRMAA, a lot of options have to be put in through the native specification (mainly unusual things that nobody ever does like requesting resources) and not all of the available options are supported by the FedStage bindings (the bundled ones don't support native spec at all). I've been seriously considering switching to calling qsub with Runtime.exec(). Depending on how complicated the jobs you want to submit are, you may have to do that or figure out the native API.
>>> If you're interesting in using the Java bindings, I can bundle them up and release them - I've been using them in development without making any changes for a while - but I'll need to sort out the copyright, licensing etc. and write something that'll build them.
>>> Scott Wilson
>>> Energy Academy
>>> EH14 4AS
>>> tel: +44 131 564 0232
>>> Scott.Wilson at Epistemy.com
>>> Epistemy Limited is a company registered in Scotland, number SC365481.
>>> Registered office: Epistemy Limited c/o Technology and Research Services, Heriot-Watt University, Riccarton, Edinburgh, EH14 4AS
>>> On 19 Oct 2010, at 00:10, Randall Svancara wrote:
>>>> Is there any documentation on torque/pbs API. I could make system calls
>>>> and run qsub and qstat, which seems to be how most people do this but I
>>>> would rather use the native API if possible.
>>>> Randall Svancara
>>>> High Performance Computing Systems Administrator
>>>> Information Technology Services
>>>> Office Telephone: (509) 335-3039
>>>> Email: rsvancara at wsu.edu
>>>> torquedev mailing list
>>>> torquedev at supercluster.org
>>> torquedev mailing list
>>> torquedev at supercluster.org
>> torquedev mailing list
>> torquedev at supercluster.org
> torquedev mailing list
> torquedev at supercluster.org
More information about the torquedev