[torqueusers] distributed make within torque
mgutteri at fhcrc.org
Thu Jul 19 18:52:37 MDT 2007
I've got a new workload for our cluster that uses make. One step of the
analysis pipeline can be run over multiple nodes, but this is dependent
on a distributed make. On SMP systems, GNU's "make -j" is the method
for parallelizing the job. On a cluster, they look to "qmake" in SGE or
"lmake" in LSF.
I haven't ever seen a distributed make for Torque- theoretically one
could be developed, but I don't know if that's within my skill set or if
it's worth the effort.
There is "pmake" for doing this over PVM, but that project appears
pretty quiet- either it's perfect or just lingering. Not sure if we'd
run into issues with newer makefiles.
There are a couple other methods I can use, but I thought I'd see if
anyone had seen or knows of a Torque-able make before I ventured forth.
Thanks in advance
More information about the torqueusers