[torqueusers] VNC service on cluster nodes with torque

Sérgio Almeida sergio.almeida at ist.utl.pt
Fri Jun 14 11:09:23 MDT 2013


On 06/14/2013 05:43 PM, Chris Hunter wrote:
> I am working on a method to provide easy remote VNC access to cluster
> nodes using regular torque "qsub" job submission. The objective is to
> allow cluster users to run GUI based software (eg. matlab, visit,
> paraview, etc.) on cluster nodes via remote access. We have run into
> several practical difficulties not directly related torque.
I think that the solution you are trying to find will lead to 
performance degradation from all your nodes throughout time. VNC 
solutions are not that clean and neat as we expect they would be by 2013.

Don't the interactive sessions with X forwarding suffice for your users 

Around here, users do qsub -I -X and they get an interactive session in 
a special queue with X11 forwarding from nodes to the entry node. Given 
that they are connected to the entry node with X11 forwarding, they will 
be able to use the graphical interfaces in their connecting computers. 
We have been using it successfully with visit, mathematica, flair, idl 
and so on. We are only allowing small sessions (< 6h) for this since 
people started leaving idle programs open from one day to the other just 
to check how it went when no time limit was set.

Regarding users that want to leave stuff running to come back later 
(implying a disconnect), we haven't thought about that yet but 
delegating the entry node window to a xpra screen should work and users 
should be able to resume it later. Check http://xpra.org/ . Also you 
should keep in mind that this leads to some problems since users will 
ask for excessive periods of time just to be able to see how it's going 
and how it went before terminating the job. This will lead to wasted 
resources on your cluster and is the main reason why our team hasn't 
endorsed the use of xpra.

Since this all works through SSH's X11's forwarding, you have all your 
security requirements fulfilled.

In the meanwhile, to save resources, we managed to use xvfb as a virtual 
X display so that users can run stuff that depends on a X11 display 
(still, that doesn't need direct interaction to start computing) without 
needing to go through the special queue to an interactive session with 
X11 forwarding. This way users run their stuff through the queue as they 
would without X11. These are the required lines right after the PBS 
header and before launching the X dependent program.

Xvfb :1 &
export DISPLAY=:1.0


> For example, users submit jobs on a public-facing server but the VNC
> service starts on a cluster node on a private network. We need to create
> a network path between the remote user and the cluster node, using the
> public-facing server as the intermediate bridge. We have found no good
> method to automate this as part of the job submission. We tested various
> port forwarding schemes that require manual intervention but nothing
> that is fully automated.
> Another issue is VNC traffic is unencrypted. We would like to encrypt
> traffic (ie. using a SSH tunnel) to the remote user. However, installing
> and configuring the required VNC & SSH client software on a remote PC is
> a support nightmare (eg. supporting unmanaged windows, mac & linux
> desktop & laptops).
> Does any manage a cluster where users submit jobs to use VNC ?
> Are you able to fully automate the VNC setup and connection ?
> Any advice on avoiding common pitfalls ? Is a fully automated VNC
> service unrealistic ?
> regards,
> chris hunter
> chris.hunter at yale.edu
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers

More information about the torqueusers mailing list