[torqueusers] Transforming node names in $PBS_NODEFILE and $PBS_GPUFILE

[Dave Ulrick]

The results of running 'osu_bw' on two of our nodes certainly suggest IB 
is being used:

# Send Buffer on HOST (H) and Receive Buffer on HOST (H)
# Size      Bandwidth (MB/s)
1                       2.42
2                       4.83
4                       9.82
8                      19.22
16                     37.55
32                     71.71
64                    143.92
128                   219.92
256                   420.21
512                   827.68
1024                 1622.58
2048                 2132.29
4096                 2508.80
8192                 2706.56
16384                2799.94
32768                3070.28
65536                3221.56
131072               3307.62
262144               3351.52
524288               3374.24
1048576              3384.84
2097152              3391.51
4194304              3393.63

Dave



On Tue, 4 Sep 2012, Lloyd Brown wrote:

> I can't speak to MVAPICH2, but I have a vague recollection that MVAPICH
> wouldn't work unless it was on IB anyway.  I could be misremembering,
> though.
>
> A good way to tell is to do a bandwidth test (eg. "osu_bw" from
> http://mvapich.cse.ohio-state.edu/benchmarks/), and see what you get.
> Generally speaking the bandwidth capabilities are different enough to
> make it pretty obvious.
>
> Lloyd Brown
-- 
Dave Ulrick
d-ulrick at comcast.net