[Mauiusers] How do I choose the network type for a job???

Lennart Karlsson Lennart.Karlsson at nsc.liu.se
Wed Feb 23 03:28:43 MST 2005


Hi Roy,

For specifying SCI interconnect in a job, we use the PBS nodes file
(/var/spool/PBS/server_priv/nodes on the PBS master node for our
configuration) like this:

n1 sci
n2 sci
n3 sci
n4 sci
n5 sci
n6 sci
n7 sci
n8 sci
n9 sci
n10 sci
n11 sci
n12 sci
n13 sci
n14 sci
n15 sci
n16 sci
n17 nosci
n18 nosci

Then it is easy to specify sci in the qsub command:

	qsub -l walltime=3:00:00 -l nodes=8:sci jobscript


Best regards,
-- Lennart Karlsson <Lennart.Karlsson at nsc.liu.se>
   National Supercomputer Centre in Linkoping, Sweden
   http://www.nsc.liu.se
   +46 706 49 55 35
   +46 13 28 26 24

Roy Dragseth <Roy.Dragseth at cc.uit.no> wrote the 19th of February: 
> We have a cluster where all nodes have Gbit and some of the nodes have SCI 
> interconnect.  I'm trying to figure out how to let users specify what network 
> they need for their jobs, after some digging in the docs I found that this 
> cfg lines should be appropriate:
> 
> #
> NODESETPOLICY     ONEOF
> NODESETATTRIBUTE  NETWORK
> NODESETDELAY      1
> NODESETLIST       gige sci
> 
> 
> #Only Gbit:
> NODECFG[compute-3-0] PARTITION=Itanium2 FEATURES=Itanium2 NETWORK=gige
> 
> #Both types
> NODECFG[compute-4-0] PARTITION=Itanium2 FEATURES=Itanium2 NETWORK=gige,sci
> 
> checknode reveals that maui has picked up these configs:
> 
> checking node compute-4-0.local
> 
> State:      Idle  (in current state for 00:00:00)
> Configured Resources: PROCS: 4  MEM: 4000M  SWAP: 4126M  DISK: 1M
> Utilized   Resources: [NONE]
> Dedicated  Resources: [NONE]
> Opsys:       DEFAULT  Arch:       linux
> Speed:      1.00  Load:       0.000
> Network:    [gige][sci]
> Features:   [Itanium2]
> Attributes: [Batch]
> Classes:    [default 4:4][pentium 4:4][itanium 4:4]
> 
> Total Time: 7:09:29:03  Up: 7:09:25:33 (99.97%)  Active: 2:17:15:39 (36.77%)
> 
> Reservations:
>   User 'itanium.0.0'(x1)  00:00:00 ->   INFINITY (  INFINITY)
>     Blocked Resources at 00:00:00    Procs: 4/4 (100.00%)
> 
> But how do I specify that a job needs nodes with the SCI interconnect???
> 
> In my job script I put the line
> 
> #PBS -Wx=NODESET:ONEOF:NETWORK:sci
> 
> but maui disregards any network specification and starts the job on whatever 
> node is available:
> 
> checking job 495
> 
> State: Running
> Creds:  user:royd  group:royd  account:uit-staff-000  class:itanium  
> qos:DEFAULT
> WallTime: 00:00:00 of 00:20:00
> SubmitTime: Sat Feb 19 00:41:19
>   (Time Queued  Total: 00:00:02  Eligible: 00:00:02)
> 
> StartTime: Sat Feb 19 00:41:21
> Total Tasks: 1
> 
> Req[0]  TaskCount: 1  Partition: Itanium2
> Network: [NONE]  Memory >= 0  Disk >= 0  Swap >= 0
> Opsys: [NONE]  Arch: [NONE]  Features: [NONE]
> Dedicated Resources Per Task: PROCS: 1  MEM: 100M
> NodeSet=ONEOF:[NONE]:sci
> Allocated Nodes:
> [compute-5-9.local:1]
> 
> 
> IWD: [NONE]  Executable:  [NONE]
> Bypass: 0  StartCount: 1
> PartitionMask: [ALL]
> Flags:       RESTARTABLE
> 
> Reservation '495' (00:00:00 -> 00:20:00  Duration: 00:20:00)
> PE:  1.00  StartPriority:  1001
> 
> 
> This is for torque-1.2.0p0 and maui 3.2.6p9.
> 
> Any hints is greatly appreciated,
> 
> r.
> -- 
> 
>   The Computer Center, University of Tromsø, N-9037 TROMSØ, Norway.
> 	      phone:+47 77 64 41 07, fax:+47 77 64 41 00
>      Roy Dragseth, High Performance Computing System Administrator
> 	 Direct call: +47 77 64 62 56. email: royd at cc.uit.no
> 
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://supercluster.org/mailman/listinfo/mauiusers
> 




More information about the mauiusers mailing list