Performance Analysis of Memory Transfers and GEMM subroutines on NVIDIA Tesla GPU Cluster
Abstract: Commodity clusters augmented with application accelerators are evolving as competitive high performance computing systems. The graphical processing unit (GPU) with very high data-level parallelism and performance per price ratio is a good target for the scientific application acceleration. In addition to the interconnect bottlenecks among the cluster compute nodes, the cost of memory copies between the host and the GPU device have to be carefully amortized to improve the overall efficiency of the application. Many scientific applications also rely on the efficient implementation of the Basic Linear Algebra Subroutines (BLAS). In this paper we study these two performance issues that are essential for porting some of the computational chemistry algorithms to the GPU cluster. A benchmark based on the NetPIPE [1] framework is developed to evaluate the latency and bandwidth of the memory copies between the host and the GPU device. The performance of the single and double matrix-multiply (GEMM) routines from the NVIDIA CUBLAS 2.0 library are studied. The results have been compared with that of the BLAS routines from the Intel math kernel library (MKL) to understand the computational trade-offs. The test bed is an Intel Xeon cluster equipped with NVIDIA Tesla GPUs.