[torqueusers] Torque on 1000 nodes ?

[Dave Jackson]

Ole,

> We typically have 100-200 jobs running, and 3 times that queued.
> With PBSPro 5.4.2 that's no sweat at all.  However, I recently
> found out that the Maui scheduler has a hard-coded limit of 4096 jobs,
> as you described.

  Maui's maximum job count is hard-coded/static but it is configured as
a single #define in a header file and the documentation describes where
and how to change this parameter.  There are sites out there running
Maui with ~15,000 jobs queued/running and Moab is being evaluated at 3x
that.  The bottleneck is usually not Maui/Moab, its typically the
resource manager.

Dave
 
On Fri, 2005-07-01 at 09:43 +0200, Ole Holm Nielsen wrote:
> Hi Garrick,
> 
> Thanks a lot.  We have two big academic clusters in Denmark that
> really need this information !
> 
> > These questions would have been a lot more interesting back in the OpenPBS
> > days :)
> 
> I quite agree.  I started to use OpenPBS in late 1999 on our
> first large Alpha cluster (http://dcwww.camp.dtu.dk/valhal.html)
> so I know about the weaknesses of OpenPBS :-)
> 
> > I can personally attest to Torque working just fine on 1700 nodes, whereas the
> > old OpenPBS code started having problems at 256 nodes.  
> 
> This is crucial information to us.  Thanks a lot !
> 
> > Overall, it's lots of jobs that are a harder problem.  Fortunately we've had
> > recent improvements in that area.  I can now have 8 thousands queued jobs and a
> > few hundred running jobs without a problem.
> 
> We typically have 100-200 jobs running, and 3 times that queued.
> With PBSPro 5.4.2 that's no sweat at all.  However, I recently
> found out that the Maui scheduler has a hard-coded limit of 4096 jobs,
> as you described.
> 
> What version of Torque do you use in order to include the "recent
> improvements" alluded to ?  What are the troubles to look out for ?
> 
> With best regards,
> Ole
> 
> Ole Holm Nielsen
> Department of Physics, Technical University of Denmark,
> Building 307, DK-2800 Kongens Lyngby, Denmark
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers