[torqueusers] log file explosion

Martin Siegert siegert at sfu.ca
Mon Oct 26 16:59:07 MDT 2009


Hi,

ever since we upgraded torque on one of our clusters to 2.3.7
the log file size (in /var/spool/torque/server_logs) exploded
to about 50 times its former size.

The logs are full of messages like these:

10/26/2009 00:00:03;0040;PBS_Server;Req;do_rpp;rpp request received on stream 9
10/26/2009 00:00:03;0040;PBS_Server;Req;do_rpp;inter-server request received
10/26/2009 00:00:03;0004;PBS_Server;Svr;is_request;message received from stream
9 (version 1)
10/26/2009 00:00:03;0004;PBS_Server;Svr;is_request;message received from stream
172.21.0.11:15003
10/26/2009 00:00:03;0004;PBS_Server;Svr;is_request;message STATUS (4) received f
rom mom on host r11 (172.21.0.11:15003) (stream 9)
10/26/2009 00:00:03;0004;PBS_Server;Svr;is_request;IS_STATUS received from r11
10/26/2009 00:00:03;0040;PBS_Server;Req;is_stat_get;received status from node r1
1
10/26/2009 00:00:03;0040;PBS_Server;Req;update_node_state;adjusting state for no
de r11 - state=0, newstate=0
10/26/2009 00:00:03;0004;PBS_Server;Svr;is_request;node 'r11' is at state '0x0'

This repeats itself for each node every second or so.
Ther settings are:
# qmgr -c 'p s' | grep log
set server log_events = 511
set server log_level = 0

However on a different cluster also running 2.3.7 no such log entries
exist - the log contains information about jobs only. On that server:
# qmgr -c 'p s' | grep log
set server log_events = 511

i.e., the log_level is not set.
I do not know whether this is relevant or not, but I cannot unset the
log_level:

# qmgr -c 'unset server log_level'
# qmgr -c 'p s' | grep log
set server log_events = 511
set server log_level = 0

Does anybody know what is causing these log entries and how I can get
rid of them (other than reverting back to 2.1.x)?

Cheers,
Martin

-- 
Martin Siegert
Head, Research Computing
WestGrid Site Lead
IT Services                                phone: 778 782-4691
Simon Fraser University                    fax:   778 782-4242
Burnaby, British Columbia                  email: siegert at sfu.ca
Canada  V5A 1S6


More information about the torqueusers mailing list