[torqueusers] Array Changes

David Beer dbeer at adaptivecomputing.com
Tue Apr 20 14:11:12 MDT 2010


Hi all

Glen Beane and I have been working together on improving arrays within TORQUE. A lot of new features have been added, and we're very excited about the improvements. I'll include the CHANGELOG below, but here are some of the highlights: 

qstat summarizes the array status.
It is possible to limit the amount of jobs that can run from a single array.
qhold, qdel, qalter, and qrls all can operate on an entire array or a range from that array.
Jobs can depend on arrays.

As well all know, we're not perfect and we could use some more testing. We'd like to invite all of you to test the new array features, if you're interested. A snapshot can be downloaded here:

http://www.clusterresources.com/downloads/torque/snapshots/array_beta.tar.gz

Here is the CHANGELOG for this build:

Array Changes
  e - TORQUE job arrays now use arrays to hold the job pointers and not linked
      lists.  This should improve performance as well as make it easier to
      implement future features we will add.
  f - Allow users to delete a range of jobs from the job array
  f - Added a slot limit to the job arrays - this restricts the number of jobs
      that can concurrently run from one job array. (add %<limit> to the end
      of the array request)
  f - added support for holding ranges of jobs from an array with a single
      qhold (using the -t option).
  f - now jobs in an array can be modified through qalter (using the -t option).
      Arrays can be modified using ranges (-t) or as a whole array.  Slot
      limits can also be modified
  f - jobs can now depend on arrays using these dependencies: afterstartarray,
      afterokarray, afternotokarray, afteranyarray, beforestartarray,
      beforeokarray, beforenotokarray, and beforeanyarray. This uses the same
      syntax as normal dependencies except the array name is used in place of a
      job name, and a number of jobs for that dependency type can be specified
      using brackets after the array name, i.e. afterokarray:<array_name>[2]
      would mean run after 2 jobs from that array finished ok.
  f - added support for using qrls on arrays, and specifying ranges with the -t
      syntax
  e - complte overhaul of job array submission code
  f - by default show only a single entry in qstat output for the whole array
      (qstat -t expands the job array)

We look forward to hearing your feedback and fixing up the code.

-- 
David Beer | Senior Software Engineer
Adaptive Computing



More information about the torqueusers mailing list