[Mauiusers] nodes going crazy

[Marcelo De Cicco]

hello!!

Week ago, we installed infiniband, since then the nodes has been crazy:

WARNING:  active job '147' has inactive node n012 allocated for 1:20:52:21
(node state: 'Down')
WARNING:  active job '142' has inactive node n012 allocated for 1:22:50:19
(node state: 'Down')
WARNING:  active job '143' has inactive node n012 allocated for 1:22:48:38
(node state: 'Down')
WARNING:  active job '144' has inactive node n012 allocated for 1:22:47:24
(node state: 'Down')
WARNING:  active job '145' has inactive node n012 allocated for 1:22:45:41
(node state: 'Down')
WARNING:  active job '146' has inactive node n012 allocated for 1:22:44:26
(node state: 'Down')
WARNING:  active job '148' has inactive node n008 allocated for 1:03:19:34
(node state: 'Down')
WARNING:  active job '150' has inactive node n008 allocated for 2:42:46
(node state: 'Down')

I restart the pbs_mom in the nodes, but nothing happens.

And suddenly , the nodes that was down, rises again!
Marcelo De Cicco

** "Antes de imprimir, pense no Meio Ambiente e nos Custos"
*
" THE MORE PROGRESS PHYSICAL SCIENCES MAKE, THE MORE THEY TEND TO ENTER
 THE DOMAIN OF MATHEMATICS, WHICH IS A KIND OF CENTRE TO WHICH THEY ALL
 CONVERGE.  WE MAY EVEN JUDGE THE DEGREE OF PERFECTION TO WHICH A
 SCIENCE HAS ARRIVED BY THE FACILITY WITH WHICH IT MAY BE SUBMITTED
 TO CALCULATION" . -- ADOLPHE QUETELET, 1796-1874
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/mauiusers/attachments/20110916/522e54f8/attachment.html