[Mauiusers] array jobs and priorities

[Steve Crusan]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Aug 8, 2011, at 4:23 PM, Andrus, Brian Contractor wrote:

> All,
>  
> I am seeing an odd thing in array job prioritization.
> It appears that when someone submits an array job with a limit on concurrent runs (eg: -t 1-1000%25), the jobs that are queued continue to accrue priority, but do not run because the limit is stopping them.
>  
> This is problematic when there are other jobs in the queue. I end up with MANY array jobs eligible to run, but do not because of the concurrency limit. Other jobs (stuck with a lower priority) do not run because they are waiting for the higher priority array jobs to run…


I'm not actually 100% sure about this, but  you can set a limit on the number idle jobs, meaning if a user submits 1000 jobs, only X number of jobs accrue priority, and the rest are blocked. I don't think blocked jobs accrue priority. If I check my queues with showq -i, idle jobs accumulate priority, but when I use showq -b, I don't see any priority listings for blocked jobs. 

http://www.adaptivecomputing.com/resources/docs/maui/6.2throttlingpolicies.php#idle

If I'm not mistake about idle jobs vs. blocked jobs in terms of priority, you can use the MAXIJOB=X parameter to set this.



>  
> Is there a way to have array jobs accrue priority at a lower rate than standard jobs? Or if a job is deferred or otherwise held off due to an array throttle, it does not accrue priority?
>  
> Finally, is there a variable for interactive jobs I can use for setting priority? I wish to have interactive jobs get the highest priority.


We map any jobs submitted to our interactive queue to a QOS, which then gives each job a starting priority. I'm starting to move towards a QOS approach with almost all of my queues, whereas TORQUE controls access to the queue via acl groups & hard limits on job geometry, and then the scheduler handles everything else through QOS.

>  
> Brian Andrus
> ITACS/Research Computing
> Naval Postgraduate School
> Monterey, California
> voice: 831-656-6238
>  
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/mauiusers

 ----------------------
 Steve Crusan
 System Administrator
 Center for Research Computing
 University of Rochester
 https://www.crc.rochester.edu/


-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJOQEjTAAoJENS19LGOpgqKvn8IAKLttBw6E+CNsjnRdoRs2aOD
20m3QwJQlQgmdXK/s0XU/evH42LCpMNUYJnMkBL3+A+hVJI8pXnhX5kLwKKsHyAf
gi+3wxBHANMHoSkEQBKVtpM9QAb9NuRmro9H9hWyHZiexipCHqGI0LLWSCy0kJsZ
kw3zw0PGP1w4Z+cGzz3baXHRFHHNOeHNOmjIZyijXWCLEWKVWCK4DAII+PweBzql
9GG6D1/OFj/fgA0g9sL7cip8KJSgwOHAEY3XNAfKEFr6F9QPQoOVSBpovHal0er5
/CTHezRmja+GkzI+WaGQdmkP3c6XuVLEbPpXwBkc8OvCZTVOCeXDMq3PMAh7lQQ=
=DQ+O
-----END PGP SIGNATURE-----