TORQUE Resource Manager

6.1 SCP Setup

To use scp based data management, TORQUE must be authorized to migrate data to any of the compute nodes. If this is not already enabled within the cluster, this can be achieved with the process described below. This process enables uni-directional access for a particular user from a source host to a destination host.

These directions were written using OpenSSH version 3.6 and may not transfer correctly to older versions.

6.1.1 - Generate SSH Key on Source Host

On the source host as the transfer user, execute the following:

>  ssh-keygen -t rsa

This will prompt for a passphrase (optional) and create two files: id_rsa and id_rsa.pub inside ~/.ssh/.

6.1.2 - Copy Public SSH Key to Each Destination Host

Transfer public key to each destination host as the transfer user:

ssh-copy-id [-i [identity_file]] [user@]machine

> scp ~/.ssh/id_rsa.pub destHost:~ (enter password)

Create an authorized_keys file on each destination host.

> ssh destHost (enter password)
> cat id_rsa.pub >> .ssh/authorized_keys

If the .ssh directory does not exist, create it with 700 privileges (mkdir .ssh;chmod 700 .ssh)

> chmod 700 .ssh/authorized_keys

6.1.3 - Configure the SSH Daemon on Each Destination Host

Some configuration of the ssh daemon may be required on the destination host. (Because this is not always the case, skip to step 4 and test the changes made to this point. If the tests fail, proceed with this step and then try testing again.) Typically, this is done by editing the /etc/ssh/sshd_config file (root access needed). To verify correct configuration, see that the following attributes are set (not commented):

RSAAuthentication    yes
PubkeyAuthentication yes

If configuration changes were required, the ssh daemon will need to be restarted (root access needed):

> /etc/init.d/sshd restart

6.1.4 - Validating Correct SSH Configuration

If all is properly configured, the following command issued on the source host should succeed and not prompt for a password:

> scp destHost:/etc/motd /tmp

Note that if this is your first time accessing destination from source, it may ask you if you want to add the fingerprint to a file of known hosts. If you specify yes, this message should no longer appear and should not interfere with scp copying via TORQUE. Also, it is important that the full hostname appear in the known_hosts file. To do this, use the full hostname for destHost, as in machine.domain.org instead of just machine.

6.1.5 - Enabling Bi-Directional SCP Access

The preceding steps allow source access to destination without prompting for a password. The reverse, however, is not true. Repeat the steps, but this time using the destination as the source, etc. to enable bi-directional SCP access (i.e. source can send to destination and destination can send to source without password prompts.)

6.1.6 - Compile TORQUE to Support SCP

In TORQUE 2.1 and later, SCP is the default remote copy protocol.

TORQUE must be re-configured (and then rebuilt) to use SCP by passing in the --with-scp flag to the configure script:

> ./configure --prefix=xxx --with-scp
> make

If special scp flags are required in your local setup, these can be specified using the rcpcmd parameter.

Troubleshooting

If, after following all of these steps, TORQUE is still having problems transferring data with scp, set the PBSDEBUG environment variable and restart the pbs_mom for details about copying. Also check the MOM log files for more details.