[torqueusers] routing queues

Sam Rash srash at yahoo-inc.com
Mon Jan 8 16:00:30 MST 2007


I asked about this quite some time back, but didn't get to fully understand
the solution.  Let's say torque has on the order of 20k+ jobs.  They are
segmented into queues which each of attributes associated with them-cpus,
memory, etc.  Most importantly, each queue has a priority.  Assume within a
queue, we want FIFO priority.


So, if we want to keep maui's footprint low (say 4k jobs), how do I use
routing queues to preserve priorities and make sure maui only gets the top
40k relevant ones?


Further, if my prioritization function in maui were much more complex, then
the entire 20k+ jobs would need to be IN maui for it to apply it's priority
function to find the top jobs, right?


More or less, we're hitting the case where we've got far more eligible jobs
to execute than nodes, but the order they go in matters immensely; in
particular assume I can even define a static order of the 20k jobs that
works, 500 new jobs could come in and may need to be anyway-middle, bottom,
or near the top.


What's the solution (Garrick) presented with routing queues?  I read docs,
but I got caught up in details and couldn't see the forest for the trees







Sam Rash

srash at yahoo-inc.com




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20070108/9f3ba168/attachment.html

More information about the torqueusers mailing list