[torqueusers] What is the Maximum no of compute Nodes in torque

Ken Nielson knielson at clusterresources.com
Wed May 13 09:26:23 MDT 2009


Gus,

Torque does not maintain open connections between the server and moms nor between the Mother Superior and sisters. The server and each mom have listen ports open where connections are made but the protocol is a request/reply so a new connection is opened for each new command and closed when the proper replies have been received. Communication also takes place over UDP, particularly on ports 15001 at the server and 15003 on sister nodes. Because UDP is connectionless PBS_NET_MAX_CONNECTIONS has no affect. 

Ken Nielson
Cluster Resources

----- Original Message -----
From: "Gus Correa" <gus at ldeo.columbia.edu>
To: "Torque Users" <torqueusers at supercluster.org>
Sent: Wednesday, May 13, 2009 8:30:30 AM GMT -07:00 US/Canada Mountain
Subject: Re: [torqueusers] What is the Maximum no of compute Nodes in torque

Hi Chi, list

Thank you!
Great information!

Yes, I read that thread while it was going on,
but I was just too lazy to search for it on the Torque list archives.

Thank you also for the information relating PBS_NET_MAX_CONNECTIONS=1024
to the maximum number of sockets, select(), etc.
I guessed there was some system related limit there, but didn't know 
which/where to find it.

In any case, according to answers on that thread, one can have more than
1024 pbs_moms running.
However, it is unclear to me how this will
work with the 1024 limit on open sockets/connections.

What happens if more than 1024 pbs_moms try to connect to the pbs_server?
Do some of them wait on a queue, or are they simply dropped,
and keep trying until the server has available connections?

Thank you,
Gus Correa
---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------

Chi Chan wrote:
> I was having the same question, and the discussion can be found in
> this discussion thread:
> 
> http://www.clusterresources.com/pipermail/torqueusers/2009-April/008978.html
> 
> Also, Linux's implementation of select() has a limit of 1024 sockets,
> so I think that's why the Torque code has PBS_NET_MAX_CONNECTIONS
> defined to 1024. If a larger number of simultaneous connections is
> needed, you will need to change the definition in Torque and also the
> system header file. (Another batch system changed the FD_SETSIZE
> definition to 8K, see:
> http://gridengine.sunsource.net/issues/show_bug.cgi?id=1502 ).
> 
> --Chi
> 
> 
> On Mon, May 11, 2009 at 2:48 AM, Rajiv Rajaian <rajiv.care at gmail.com> wrote:
>> Hi
>> Im using torque-2.0.0p8.
>> What is the maximum no of the compute nodes that can be configured with
>> torque..
>> Is there any configuration to limit the no of compute nodes that can be
>> configured..
>>
>> Thanks in Advance
>> Rajiv
>> Project associate,
>> CARE,
>> Anna university
>>
>> _______________________________________________
>> torqueusers mailing list
>> torqueusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>
>>
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers

_______________________________________________
torqueusers mailing list
torqueusers at supercluster.org
http://www.supercluster.org/mailman/listinfo/torqueusers


More information about the torqueusers mailing list