[torqueusers] Torque and Propack 5 cpusets.

Adam Steenwyk ajamess at umich.edu
Wed Aug 8 08:14:02 MDT 2007


Glad to see some nice info going around about this, it's very  
helpful!  Martin, I am assuming your set up simply allows for  
interactive logins on the same machine that jobs are running on?

In our setup, something similar to your approach would be very nice;  
however, the real gravy comes in being able to carve off cpusets for  
individual jobs.  I spent a few hours looking at the cpuset code  
(mostly in start_exec.c), and came to the conclusion that before PP5  
cpusets could be used in conjunction with torque a rather large code  
rewrite would be in order, since AFAIK the torque snaps are based off  
of PP3.

Here are some links showing the large differences in the PP3/5 APIs.

* Propack3: http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi? 
coll=0650&db=man&fname=/usr/share/catman/p_man/cat5/ 
cpuset.z&srch=cpusets
* Propack5: http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi/ 
linux/bks/SGI_Admin/books/LX_Resource_AG/sgi_html/apa.html

That being said, does anyone have any more information regarding  
running jobs within cpusets with PP5?

Thanks everyone!

Adam.
CAC - University of Michigan

On Aug 8, 2007, at 9:56 AM, Rushton Martin wrote:

> |
> |the big question is whether an individual CPUset is created by
> |pbs_mom for each job or not. Reserving some dedicated
> |resources for system processes or interactive work of course
> |is a first step - but what you usually want are dedicated
> |resources (in terms of fixed CPUs and fixed "local" memory)
> |for all individual PBS jobs.
> |It's not clear to me if (how) you configured the latter or not.
> |torque-2.2-(snapshots) might be able to do that somehow ...
> |
> |
> |Regards,
> |
> |thomas
>
> No, as I said in my earlier mail it is the simple approach.  All  
> jobs and the daemons run in /batch, therefore so do their children,  
> the jobs.  I don't (currently) attempt to create the cpusets "on  
> the fly" for each job.  As you say, a first step, but one that  
> works for us at the moment.
>
> Martin Rushton
> HPC Facility
>
> QinetiQ
> Bldg H4 Rm 6
> MoD Fort Halstead
> Sevenoaks
> Kent, TN14 7BP
>
> Tel:    01959 514777
> Email:  jmrushton at QinetiQ.com
> Mobile: 07939 219057
> Fax:    01959 51 6050
> Web:    www.QinetiQ.com
>
> QinetiQ - The Global Defence and Security Experts
>
> The information contained in this E-Mail and any subsequent
> correspondence is private and is intended solely for the intended
> recipient(s).  The information in this communication may be
> confidential and/or legally privileged.  Nothing in this e-mail is
> intended to conclude a contract on behalf of QinetiQ or make QinetiQ
> subject to any other legally binding commitments, unless the e-mail
> contains an express statement to the contrary or incorporates a  
> formal Purchase Order.
>
> For those other than the recipient any disclosure, copying,
> distribution, or any action taken or omitted to be taken in reliance
> on such information is prohibited and may be unlawful.
>
> Emails and other electronic communication with QinetiQ may be
> monitored and recorded for business purposes including security, audit
> and archival purposes.  Any response to this email indicates consent
> to this.
>
> Telephone calls to QinetiQ may be monitored or recorded for quality
> control, security and other business purposes.
>
> QinetiQ Limited
> Registered in England & Wales: Company Number:3796233
> Registered office: 85 Buckingham Gate, London SW1E 6PD, United Kingdom
> Trading address: Cody Technology Park, Cody Building, Ively Road,  
> Farnborough, Hampshire, GU14 0LX, United Kingdom
> http://www.QinetiQ.com/home/legal.html
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers



More information about the torqueusers mailing list