[torqueusers] python pbs_submit

[nathaniel.x.woody at gsk.com]

Nate,

As a gigantic fan of pbs_python, we have a library that wraps that 
interface for job_submission (etc) and here are the funtions we use for 
constructing the attrl and attropls.  I'm sure Bas can give a better way 
to do this, but we use the pbs.new_attrl function to create an attrl 
struct with as many slots as we need.  Then we plug in the name, resource, 
and stringed value to each of the empty slots.  Then we can just attach 
this to the job (for instance attropls get added at submit job).  This 
works for creating both attrls and attropls, just pass the correct 
function pointer to make_attr_structs.  The pbs_call can safely be 
replaced with the actual function call, we just route all of our actual 
Torque communication through a single channel for managing 
connections/errors, etc.

def make_attr_structs (attributes, constructor):
  """Create an array of structs (via the specified 'constructor') which 
encode
     the specified 'attributes'."""

  # Determine the number of structs evinced by 'attributes'.
  attr_count = len(attributes)
 
  #replace the below with something like:
  #constructor(attr_count)
  attr  = pbs_call(constructor, attr_count)
  index = 0

  # Pack the struct array.
  for pair in attributes.iteritems():
    name, ds = pair

    # If 'ds' is a dictionary, then treat it as containing valued 
resources.
    if type(ds) == types.DictType:
      for resource_pair in ds.iteritems():
        resource, value       = resource_pair
        attr[index].name      = name
        attr[index].resource  = resource
        attr[index].value     = str(value)
        index                += 1
 
    else:
      # If 'ds' is a scalar object, then wrap a list around it.
      if type(ds) != types.ListType:
        ds = [ ds ]

      attr[index].name  = name
      attr[index].value = ",".join(map(lambda x: str(x), ds))
      index += 1

  return attr

def make_attrl (attributes):
  """Obtain an array of 'attrl' structs which encode the specified 
'attributes'.
  """

  return make_attr_structs(attributes, constructor = pbs.new_attrl)

def make_attropl (attributes):
  """Obtain an array of 'attropl' structs which encode the specified
     'attributes'."""

  return make_attr_structs(attributes, constructor = pbs.new_attropl)

Hope this helps.  If not let me know.

Best,
Nate







"Nate Coraor" <nate at psu.edu> 
Sent by: torqueusers-bounces at supercluster.org
05-Sep-2006 13:42
 
To

cc

Subject
[torqueusers] python pbs_submit






Hello,

I'm using the pbs python module (from sara.nl) to submit jobs to 
torque.  Everything works well except that I can't generate an attrl, 
in order to pass job parameters to pbs_submit.  The swig-generated 
code is very hard to follow and there doesn't seem to be any 
documentation or examples of using an attrl.  If anyone can provide a 
tip on how to create this struct in python, I'd greatly appreciate it.

Thanks,
--nate
_______________________________________________
torqueusers mailing list
torqueusers at supercluster.org
http://www.supercluster.org/mailman/listinfo/torqueusers


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20060906/3f5ad7a5/attachment.html