[torqueusers] VNC service on cluster nodes with torque
sergio.almeida at ist.utl.pt
Fri Jun 14 11:09:23 MDT 2013
On 06/14/2013 05:43 PM, Chris Hunter wrote:
> I am working on a method to provide easy remote VNC access to cluster
> nodes using regular torque "qsub" job submission. The objective is to
> allow cluster users to run GUI based software (eg. matlab, visit,
> paraview, etc.) on cluster nodes via remote access. We have run into
> several practical difficulties not directly related torque.
I think that the solution you are trying to find will lead to
performance degradation from all your nodes throughout time. VNC
solutions are not that clean and neat as we expect they would be by 2013.
Don't the interactive sessions with X forwarding suffice for your users
Around here, users do qsub -I -X and they get an interactive session in
a special queue with X11 forwarding from nodes to the entry node. Given
that they are connected to the entry node with X11 forwarding, they will
be able to use the graphical interfaces in their connecting computers.
We have been using it successfully with visit, mathematica, flair, idl
and so on. We are only allowing small sessions (< 6h) for this since
people started leaving idle programs open from one day to the other just
to check how it went when no time limit was set.
Regarding users that want to leave stuff running to come back later
(implying a disconnect), we haven't thought about that yet but
delegating the entry node window to a xpra screen should work and users
should be able to resume it later. Check http://xpra.org/ . Also you
should keep in mind that this leads to some problems since users will
ask for excessive periods of time just to be able to see how it's going
and how it went before terminating the job. This will lead to wasted
resources on your cluster and is the main reason why our team hasn't
endorsed the use of xpra.
Since this all works through SSH's X11's forwarding, you have all your
security requirements fulfilled.
In the meanwhile, to save resources, we managed to use xvfb as a virtual
X display so that users can run stuff that depends on a X11 display
(still, that doesn't need direct interaction to start computing) without
needing to go through the special queue to an interactive session with
X11 forwarding. This way users run their stuff through the queue as they
would without X11. These are the required lines right after the PBS
header and before launching the X dependent program.
Xvfb :1 &
> For example, users submit jobs on a public-facing server but the VNC
> service starts on a cluster node on a private network. We need to create
> a network path between the remote user and the cluster node, using the
> public-facing server as the intermediate bridge. We have found no good
> method to automate this as part of the job submission. We tested various
> port forwarding schemes that require manual intervention but nothing
> that is fully automated.
> Another issue is VNC traffic is unencrypted. We would like to encrypt
> traffic (ie. using a SSH tunnel) to the remote user. However, installing
> and configuring the required VNC & SSH client software on a remote PC is
> a support nightmare (eg. supporting unmanaged windows, mac & linux
> desktop & laptops).
> Does any manage a cluster where users submit jobs to use VNC ?
> Are you able to fully automate the VNC setup and connection ?
> Any advice on avoiding common pitfalls ? Is a fully automated VNC
> service unrealistic ?
> chris hunter
> chris.hunter at yale.edu
> torqueusers mailing list
> torqueusers at supercluster.org
More information about the torqueusers