[torqueusers] Submission number limits?

[James J Coyle]

Garrick,

  Syntax correction:

create queue execq  queue_type = E, max_queuable=1000

(No between queu  and able  )

  Thanks for the heads up about this, I needed it too.


-- 
 James Coyle, PhD
 SGI Origin, Alpha, Xeon and Opteron Cluster Manager
 High Performance Computing Group     
 235 Durham Center            
 Iowa State Univ.          
 Ames, Iowa 50011           

> 
> --===============1321683781==
> Content-Type: multipart/signed; micalg=pgp-sha1;
> 	protocol="application/pgp-signature"; boundary="sAMF9MHkXj43A5i+"
> Content-Disposition: inline
> 
> 
> --sAMF9MHkXj43A5i+
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: inline
> Content-Transfer-Encoding: quoted-printable
> 
> On Wed, May 07, 2008 at 02:53:04PM -0500, Jeremy Mann alleged:
> > Good afternoon all, I have one user that wants to submit roughly 140,000
> > jobs to our queue. We tried it last week and it never worked. It took
> > nearly an hour to submit all of them, then the PBS scheduler would stop
> > responding and give:
> >=20
> > 05/02/2008 14:39:50;0100; pbs_sched;Req;;Leaving schedule
> >=20
> > 05/02/2008 14:39:50;0080; pbs_sched;Svr;main;brk point 760373248
> > 05/02/2008 14:39:53;0100; pbs_sched;Req;;Entering Schedule
> > 05/02/2008 14:42:53;0002; pbs_sched;Svr;toolong;alarm call
> >=20
> > The jobs are quite small and they run for about a minute. Now we're
> > thinking about breaking them up into 100 or 1000 job chunks.
> >=20
> > I'm curious if the number of jobs being submitted, in our case 140,000, is
> > too large for PBS/Torque to handle.
> >=20
> > Torque 2.1.2 x86_64 and the built in scheduler (not MAUI)
> 
> The trick is to limit the number of jobs visible to the scheduler by using a
> routing queue to spool jobs into the execution queue.
> 
> So you do something like this:
> 
> create queue spoolq queue_type =3D Route, route_destinations =3D execq
> create queue execq  queue_type =3D E, max_queueable=3D1000
> 
> Submit jobs to spoolq and it should handlelarge numbers of jobs.
> 
> --=20
> Garrick Staples, GNU/Linux HPCC SysAdmin
> University of Southern California
> 
> Please avoid sending me Word or PowerPoint attachments.
> See http://www.gnu.org/philosophy/no-word-attachments.html
> 
> --sAMF9MHkXj43A5i+
> Content-Type: application/pgp-signature
> Content-Disposition: inline
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
> 
> iD8DBQFIIg4p0SBUxJbm9HMRAo+CAKCiRyKy/ZiotJ9LVc9V1QOpeTGU4ACdEzdC
> mIw0jjIngLp9eb785EZQJ1I=
> =Hz5m
> -----END PGP SIGNATURE-----
> 
> --sAMF9MHkXj43A5i+--
> 
> --===============1321683781==
> Content-Type: text/plain; charset="us-ascii"
> MIME-Version: 1.0
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
> 
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers
> 
> --===============1321683781==--
>