[torqueusers] Job Input Files
download at hpc.unm.edu
Thu Feb 11 14:18:46 MST 2010
I'm not entirely sure you're proposing here: Are you proposing to
set up STDIN for the job script (the pbs script that you submit to the
batch server which is subsequently run on the "lead node" of your job)?
If so, in my experience, this proposed feature is not currently missed.
I have never once missed not having STDIN for my job script: In general,
they are pretty simple things that end up starting the "real application"
on all of the nodes/processors I've requested. Almoast always they look
mpirun ... program_or_script
Its this program_or_script that sometimes needs input from STDIN. How we
generally handle that is to write a small "wrapper script" that sets up
the proper STDIN redirection. N.B. I find that if the program being run
needs input from STDIN, it is common that the different invocations of
said program (i.e. each process in an MPI job) may require different input
James E. Prewett Jim at Prewett.org download at hpc.unm.edu
Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/
Designated Security Officer OpenPGP key: pub 1024D/31816D93
HPC Systems Engineer III UNM HPC 505.277.8210
On Thu, 11 Feb 2010, David Beer wrote:
> We are interested in doing something a little more than staging-in files - we would go the next step and pipe the file to stdin. Since we know that doing this would include making major changes to TORQUE, we're interested in finding out if what is currently available is working for people.
> >From our experience, despite shared filesystems, people are still interested in having features like the one described above.
> ----- "Garrick Staples" <garrick at usc.edu> wrote:
> > On Thu, Feb 11, 2010 at 10:26:33AM -0700, David Beer alleged:
> > > Hi all,
> > >
> > > We are considering adding support for job input files in TORQUE. I'm
> > wondering what work-arounds (if-any) are you all using to get input
> > files to your jobs?
> > Torque already has a stagein facility.
> > These days, most sites have shared filesystems which makes
> > stagein/stageout
> > somewhat obsolete.
> > Most users just read their files from their shared filesystem; perhaps
> > making
> > local copies in /tmp for faster access.
> > --
> > Garrick Staples, GNU/Linux HPCC SysAdmin
> > University of Southern California
> > Life is Good!
> > _______________________________________________
> > torqueusers mailing list
> > torqueusers at supercluster.org
> > http://www.supercluster.org/mailman/listinfo/torqueusers
> David Beer | Senior Software Engineer
> Adaptive Computing
> torqueusers mailing list
> torqueusers at supercluster.org
More information about the torqueusers