[torqueusers] Running torque with iptables

[Michael Jennings]

On Wednesday, 19 October 2011, at 16:09:48 (-0500),
Ti Leggett wrote:

> We're rolling out locking down machines much more tightly using
> iptables after a security incident. I've read the documentation and
> I have tcp/udp 15001 and tcp 15004 open on the PBS server, I have
> tcp 15002, tcp/udp 15003 and udp 0-1023 opened on the PBS MOMs and I
> have udp 0-1023 on the submit hosts. However it seems the MOM
> superior is trying to talk back to the submit host on tcp ephemeral
> ports 1024. Is there any way to restrict the range of those ports
> it's trying to use so that I can open those up appropriately, or am
> I going to have to take the (undesired) route of opening everything
> up between the MOMs and submit hosts?

In src/cmds/qsub.c, function interactive_port(), the following code
determines that the port number will be arbitrary for the interactive
job listener:

  myaddr.sin_port = 0;

Two possible solutions here:  If you know only 1 qsub -I will ever be
running on a particular node at any one time, you can hardcode the
port here by changing 0 to htons(12345) (or whatever port number you
choose).

The better solution is going to wrap the bind() in a for loop to try a
range of port numbers consecutively until the bind() succeeds (or you
run out of ports).

  for (port = LOW_PORT; port <= HIGH_PORT; port++) {
    myaddr.sin_port = htons(port);
    if (bind(*sock, (struct sockaddr *)&myaddr, namelen) >= 0) {
      break;
    }
  }
  if (port > HIGH_PORT) {
    perror("qsub: unable to bind to socket");
    exit(1);
  }

Something like that.

If that works, you (or someone) might be inclined to add a
configuration option to specify the port range.  :-)

HTH,
Michael

-- 
Michael Jennings <mej at lbl.gov>
Linux Systems and Cluster Engineer
High-Performance Computing Services
Bldg 50B-3209E      W: 510-495-2687
MS 050C-3396        F: 510-486-8615