[torqueusers] Semaphores limit per job/user in torque?

Andrew Savchenko bircoph at gmail.com
Sat Sep 21 15:09:51 MDT 2013


Hello,

is it possible to limit or isolate semaphores per job or user at
worker node in torque?

At our cluster we have a problem with buggy user jobs which left
semaphores behind leading to semaphore limit exhaustion. While limit
may be lifted, this is not a proper solution since it will be reached
again later. ATM we a running cron job using some heuristics to
determine which semaphores are safe to clear. But this is still
nothing but a workaround.

The proper way is to isolate job or at least user IPC namespace on
nodes. This can be done using IPC namespace kernel feature, though I
don't know if torque is capable of this or any other ways to control
job's IPC.

ATM we're using torque-3.0.6, though if 4.x branch is capable of this
feature, it will be a good reason to migrate.

Best regards,
Andrew Savchenko
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/torqueusers/attachments/20130922/b2160b66/attachment.bin 


More information about the torqueusers mailing list