[torqueusers] Limit max jobs submitted

glen.beane at gmail.com glen.beane at gmail.com
Tue Nov 22 18:24:55 MST 2011




On Nov 22, 2011, at 5:45 PM, Martin Siegert <siegert at sfu.ca> wrote:

> Hi Brian,
> 
> On Tue, Nov 22, 2011 at 09:28:56PM +0000, Andrus, Brian Contractor wrote:
>> They do work, but they do not do what I need.
>> 
>> See when someone submits >100000 array jobs, it fills up the job list that is used to schedule the jobs.
>> 
>> That is MAXJOB tells how many jobs to work with within moab to decide priority and who to run. So if MAXJOB is set to 50000, and someone submits an array of 100000, then 1/2 of their jobs get pulled in and the rest are ignored (for now) by moab.
>> Now along comes supersensitive.user who submits his interactive job, which will sit for way too long because moab isn't even going to schedule it. In fact, moab is ignoring it.
>> 
>> I could set MAXJOB to 500000, but that still doesn't prevent a user from submitting too many jobs such that the list that is looked at does not over-fill.
>> 
>> Is there a setting were if someone were to submit >X jobs (array or otherwise), torque/moab will not even allow it in?
>> 
>> 
>> Brian Andrus
>> ITACS/Research Computing
>> Naval Postgraduate School
>> Monterey, California
>> voice: 831-656-6238
> 
> I ran into exactly the same problem a few weeks ago.
> Currently the only way to prevent a user from overloading moab and thus
> preventing it from scheduling jobs in priority order is to 
> 
> 1) set MAXJOB to some value X
> 2) use
> 
> set queue exec max_user_queuable = Y
> 
>   for the execution queues AND additionally set
> 
> set queue rte max_user_queuable = Y
> 
>   for ALL routing queues that route jobs to the relevant execution queues.
> 
> Y must be much smaller than X.
> Unfortunately, there currently is no limit like
> 
> set server max_user_queuable = Y
> 


This is a reasonable feature request. I have been thinking about implementing it for a few years...




> which would be a more logical way of preventing this denial-of-service
> attack against moab.
> 
> Cheers,
> Martin
> 
> -- 
> Martin Siegert
> Simon Fraser University
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers


More information about the torqueusers mailing list