[torqueusers] distributed make within torque

Michael Gutteridge mgutteri at fhcrc.org
Thu Jul 19 18:52:37 MDT 2007


I've got a new workload for our cluster that uses make.  One step of the
analysis pipeline can be run over multiple nodes, but this is dependent
on a distributed make.  On SMP systems, GNU's "make -j" is the method
for parallelizing the job.  On a cluster, they look to "qmake" in SGE or
"lmake" in LSF.

I haven't ever seen a distributed make for Torque- theoretically one
could be developed, but I don't know if that's within my skill set or if
it's worth the effort.

There is "pmake" for doing this over PVM, but that project appears
pretty quiet- either it's perfect or just lingering.  Not sure if we'd
run into issues with newer makefiles.

There are a couple other methods I can use, but I thought I'd see if
anyone had seen or knows of a Torque-able make before I ventured forth.

Thanks in advance

Michael

  



More information about the torqueusers mailing list