[torqueusers] PBS RM extensions usage
jay at nersc.gov
Fri Nov 21 00:56:41 MST 2008
Thanks for the clarification. I did not realize there was a hierarchy to
the node access policy and it looks like Maui obeys the same rules as
Moab in that regard. The CLASSCFG setting does not work for Maui,
unfortunately, but I was able to get what I want by reversing the
problem -- i.e. setting the default NODEACCESSPOLICY to be SHARED and
then setting SINGLEJOB only for all the parallel jobs via the submit
filter (by adding the RM extension).
On 11/19/2008 2:15 PM, Craig West wrote:
> I was investigating this very issue myself recently. The documentation
> is pretty clear (when you find the right place) in that you can only
> increase the restrictions, SHARED is the lowest level.
> Read the "Note" located here:
> One possible option, although I'm not sure it is currently available
> MAUI, would be to setup different queues with different policies.
> The MOAB documentation says this can be done with the following setting:
> CLASSCFG[batch] NODEACCESSPOLICY=SINGLEJOB
> I am running MAUI, not MOAB, but have not tested this function.
> On 11/19/2008 12:12 PM, Jay Srinivasan wrote:
>> I'm trying to use the RM extensions with Maui 3.2.6p20 and Torque
>> 2.3.0. My default setting in Maui for node allocation parameters is
>> as follows:
>> NODEACCESSPOLICY SINGLEJOB
>> NODEAVAILABILITYPOLICY DEDICATED
>> NODEALLOCATIONPOLICY MINRESOURCE
>> But, I'd like to submit a job to a serial queue that allows for SHARED
>> node access, and so I submit a job as follows:
>> qsub -Wx=NACCESSPOLICY:SHARED serial.job
>> where I request -lnodes=1:ppn=1
>> But this seems to be getting ignored and Maui continues to assume the
>> node access policy is SINGLEJOB. The log file shows the following
>> (where it seems to show that it accepts the RM extension but ignores it).
>> 11/18 12:10:36 INFO: PBS attribute 'submit_args' value:
>> '-Wx=NACCESSPOLICY:SHARED serial.job ' (r: NULL)
>> 11/18 12:10:36 INFO: PBS attribute 'x' value:
>> 'NACCESSPOLICY:SHARED' (r: NULL)
>> 11/18 12:10:36 MJobSetAttr(1493,RMXString,Value,1,0)
>> 11/18 12:10:36 MJobProcessExtensionString(1493,NACCESSPOLICY:SHARED)
>> 11/18 12:10:36 MUGetIndex(NACCESSPOLICY:SHARED,ValList,0)
>> 11/18 12:10:36 MUGetIndex(SHARED,ValList,2)
>> 11/18 12:10:36 INFO: job node access policy set to SHARED
>> 11/18 12:10:36 MJobSetCreds(1493,jay,mpccc,)
>> 11/18 12:10:36 MUserAdd(UName,UP)
>> 11/18 12:10:36 MGroupAdd(GName,GP)
>> 11/18 12:10:36 MJobGetAccount(1493,A)
>> 11/18 12:10:36 INFO: job flags for job 1493: 800, req
>> 11/18 12:10:36 MJobSetAttr(1493,GAttr,Value,1,5)
>> 11/18 12:10:36 MQOSGetAccess(1493,NULL,QAL,QDef)
>> My question is whether the RM extension for NACCESSPOLICY actually
>> work? Am I missing some other setting (in Torque, for instance) that
>> needs to be done for this to work? I've tried the same syntax with
>> other RM extensions (HOSTLIST, for example), and those work, but
>> NACCESSPOLICY does not.
>> Thanks for any help.
More information about the torqueusers