[torqueusers] Strange problem in Torque and Maui

Kandy Wong kandyw at triumf.ca
Tue Oct 23 15:31:30 MDT 2007


Hi Everyone,

I'm using torque-2.1.8 with maui-3.2.6p17.
The system worked fine before but days ago when I tried to submit a job, 
the job never gets run even though I'm sure all the nodes are 
available.  But I still could force the job to run using 'qrun'.
The strange things are:
when I try to 'showq', it shows
0 of    0 Processors Active (0.00%)

I can use the command 'pbsnodes -a', qstat' and 'qmgr' on the nodes.  
But not on the server. The followings are the output on the server:
pbsnodes -a
No default server name.
pbsnodes: cannot connect to server , error=15034

qstat:
No default server name.
qstat: cannot connect to server (null) (errno=15034)

qmgr
No default server name.
qmgr: cannot connect to server

Also, when I use the 'xpbs' and 'xpbsmon' commands on the nodes, it 
shows all the correction information like the server name and queues.
But when I tried it on the server, it complains about 'No 
Permission.\nxpbs_datadump: Can not connect to server (15007)'

So I look at the /var/spool/pbs/server_logs:
PBS_Server;Svr;WARNING;ALERT: unable to contact node server name
PBS_Server;Svr;PBS_Server;Connection refused (111) in contact_sched, 
Could not contact Scheduler - port 15004

And in the log /var/spool/maui/
ERROR:    cannot connect to PBS server 'server name'  rc: -1 (errno: 15007)
ALERT:    cannot re-initialize PBS interface
10/23 11:12:02 ALERT:    cannot load cluster resources on RM (RM '0' 
failed in function 'clusterquery')
10/23 11:12:02 WARNING:  no resources detected

The only thing I can think of is that the server got reset a week ago 
but I'm sure all the pbs_server, pbs_mom and maui services are back on.  
'server_name' is in /var/spool/pbs
Any ideas?
Thank you very much.

Kandy



More information about the torqueusers mailing list