[torqueusers] Question about file spooling, spool directory

[Craig Tierney]

I used to use OpenPBS ages ago, and I am looking to install Torque
on my cluster.  I see that Torque still has the behavior that the
standard output files (stdout and stderr) are still spooled and
then copied only when the job is completed.  I understand that when
PBS was created, that a shared filesystem was not necessary a guarantee,
and the copy was needed.

Now, I don't see why this function would be needed.  NFS can scale quite
well, and there are many distributed filesystems (Lustre, Panasas, etc.)
that can scale so that all nodes have a uniform shared filesystem.

So, why does Torque still do the spool and copy method?  With Torque,
you can spool the files into the users home directory (with --disable-spool),
although the filenames are still the spooled names, not the final name.
Is there a fundamental technological reason (besides no one has
written the code) for not just putting the stderr and stdout into the "final"
location of that data?

Craig

-- 
Craig Tierney (craig.tierney at noaa.gov)