[torqueusers] Question about file spooling, spool directory
Craig.Tierney at noaa.gov
Mon Nov 10 13:54:19 MST 2008
I used to use OpenPBS ages ago, and I am looking to install Torque
on my cluster. I see that Torque still has the behavior that the
standard output files (stdout and stderr) are still spooled and
then copied only when the job is completed. I understand that when
PBS was created, that a shared filesystem was not necessary a guarantee,
and the copy was needed.
Now, I don't see why this function would be needed. NFS can scale quite
well, and there are many distributed filesystems (Lustre, Panasas, etc.)
that can scale so that all nodes have a uniform shared filesystem.
So, why does Torque still do the spool and copy method? With Torque,
you can spool the files into the users home directory (with --disable-spool),
although the filenames are still the spooled names, not the final name.
Is there a fundamental technological reason (besides no one has
written the code) for not just putting the stderr and stdout into the "final"
location of that data?
Craig Tierney (craig.tierney at noaa.gov)
More information about the torqueusers