[Mauiusers] How do I choose the network type for a job???

[Roy Dragseth]

Hi.

We have a cluster where all nodes have Gbit and some of the nodes have SCI 
interconnect.  I'm trying to figure out how to let users specify what network 
they need for their jobs, after some digging in the docs I found that this 
cfg lines should be appropriate:

#
NODESETPOLICY     ONEOF
NODESETATTRIBUTE  NETWORK
NODESETDELAY      1
NODESETLIST       gige sci


#Only Gbit:
NODECFG[compute-3-0] PARTITION=Itanium2 FEATURES=Itanium2 NETWORK=gige

#Both types
NODECFG[compute-4-0] PARTITION=Itanium2 FEATURES=Itanium2 NETWORK=gige,sci

checknode reveals that maui has picked up these configs:

checking node compute-4-0.local

State:      Idle  (in current state for 00:00:00)
Configured Resources: PROCS: 4  MEM: 4000M  SWAP: 4126M  DISK: 1M
Utilized   Resources: [NONE]
Dedicated  Resources: [NONE]
Opsys:       DEFAULT  Arch:       linux
Speed:      1.00  Load:       0.000
Network:    [gige][sci]
Features:   [Itanium2]
Attributes: [Batch]
Classes:    [default 4:4][pentium 4:4][itanium 4:4]

Total Time: 7:09:29:03  Up: 7:09:25:33 (99.97%)  Active: 2:17:15:39 (36.77%)

Reservations:
  User 'itanium.0.0'(x1)  00:00:00 ->   INFINITY (  INFINITY)
    Blocked Resources at 00:00:00    Procs: 4/4 (100.00%)

But how do I specify that a job needs nodes with the SCI interconnect???

In my job script I put the line

#PBS -Wx=NODESET:ONEOF:NETWORK:sci

but maui disregards any network specification and starts the job on whatever 
node is available:

checking job 495

State: Running
Creds:  user:royd  group:royd  account:uit-staff-000  class:itanium  
qos:DEFAULT
WallTime: 00:00:00 of 00:20:00
SubmitTime: Sat Feb 19 00:41:19
  (Time Queued  Total: 00:00:02  Eligible: 00:00:02)

StartTime: Sat Feb 19 00:41:21
Total Tasks: 1

Req[0]  TaskCount: 1  Partition: Itanium2
Network: [NONE]  Memory >= 0  Disk >= 0  Swap >= 0
Opsys: [NONE]  Arch: [NONE]  Features: [NONE]
Dedicated Resources Per Task: PROCS: 1  MEM: 100M
NodeSet=ONEOF:[NONE]:sci
Allocated Nodes:
[compute-5-9.local:1]


IWD: [NONE]  Executable:  [NONE]
Bypass: 0  StartCount: 1
PartitionMask: [ALL]
Flags:       RESTARTABLE

Reservation '495' (00:00:00 -> 00:20:00  Duration: 00:20:00)
PE:  1.00  StartPriority:  1001


This is for torque-1.2.0p0 and maui 3.2.6p9.

Any hints is greatly appreciated,

r.
-- 

  The Computer Center, University of Tromsø, N-9037 TROMSØ, Norway.
	      phone:+47 77 64 41 07, fax:+47 77 64 41 00
     Roy Dragseth, High Performance Computing System Administrator
	 Direct call: +47 77 64 62 56. email: royd at cc.uit.no