[torquedev] Re: [torqueusers] pbs_server: socket_to_handle, internal socket table full

[Chris Samuel]

----- "Chris Samuel" <csamuel at vpac.org> wrote:

> Anyone seeing these messages popping up in syslog ?
>
> pbs_server: socket_to_handle, internal socket table full

I can confirm that this is happening with 2.3.1-snap.200804211148
as well as the stock 2.3.0.

It looks like if pbs_server is trying to talk to a mom
which accepts connection but never does anything with it
(due to the node locking up) that connection doesn't drop
out.

In our case pbs_server has 80 connections in "established"
state to a problem node, e.g.:

pbs_serve 3551 root  231u  IPv4          748982477               TCP 172.17.1.254:876->172.17.1.63:15002 (ESTABLISHED)
pbs_serve 3551 root  236u  IPv4          748987070               TCP 172.17.1.254:835->172.17.1.63:15002 (ESTABLISHED)
pbs_serve 3551 root  245u  IPv4          748982948               TCP 172.17.1.254:890->172.17.1.63:15002 (ESTABLISHED)

cheers,
Chris
-- 
Christopher Samuel - (03) 9925 4751 - Systems Manager
 The Victorian Partnership for Advanced Computing
 P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency