[torquedev] [torqueusers] Performance of non-GPU codes on GPU nodes reduced by nvidia-smi overhead
djohnson at osc.edu
Wed Feb 22 14:50:32 MST 2012
I'm in the process of devising a cleaner way to do what I did in my
previous quick and dirty patch. I'm working out of the torque-2.5
branch. I'd be very interested in working with whoever gets assigned
this problem at Adaptive.
And as an update, I've executed many jobs on the GPU and non-GPU nodes
with the patch referenced below. Same mom binary with nvml support on
both types of nodes with no apparent problems.
At Wed, 22 Feb 2012 13:53:16 -0700,
David Beer wrote:
> [1 <multipart/alternative (7bit)>]
> [1.1 <text/plain; ISO-8859-1 (7bit)>]
> [1.2 <text/html; ISO-8859-1 (quoted-printable)>]
> This is now officially in the list of things to do, and I'll keep you updated on it. That may sound
> terrible, but it should be done quickly.
> On Fri, Feb 17, 2012 at 3:12 PM, Doug Johnson <djohnson at osc.edu> wrote:
> At Fri, 17 Feb 2012 13:10:41 -0700,
> David Beer wrote:
> > [1 <multipart/alternative (7bit)>]
> > [1.1 <text/plain; ISO-8859-1 (7bit)>]
> > [1.2 <text/html; ISO-8859-1 (quoted-printable)>]
> > Doug,
> > I have created a ticket for our documentation team to note that the TDK is where nvml.h can be
> > We also thank you for the patch. I believe there is some more work that needs to be done beyond
> > this change, but we will look to get those done very soon. I think it would be ideal to allow
> people to
> > use the same binary for both GPU enabled and non-GPU enabled nodes.
> Yeah, conceptual versus actually working. There's no really proper
> way to do this with how the gpu code is currently inline with many
> ifdefs. It's a surprisingly small number of entry points that need to
> be modified. See the attached patch, this allows an NVML enabled
> build to run on either a GPU or non-GPU node. If all the GPU routines
> were moved into their own file this could be done cleanly and without
> a lot of effort.
> PS. caveat emptor with the patch, I've run two jobs on the nodes so
> it's not exactly thought out.
> PPS. Should move thread to torque devel, sorry.
> torqueusers mailing list
> torqueusers at supercluster.org
> David Beer | Software Engineer
> Adaptive Computing
> [2 <text/plain; us-ascii (7bit)>]
> torquedev mailing list
> torquedev at supercluster.org
More information about the torquedev