Purpose: To implement and measure performance of Matrix Multiply-Accumulate(like D = A * B + C) on CPU, GPU (with/without Tensor Cores), respectively.

Note that this repository only contains the less performant version of implementations. It is designed for demonstration purposes only to show how your project should be done.

includes sample code of MMA with a single thread on CPU

includes sample code of MMA on GPU without Tensor Cores by CUDA API

includes sample code of MMA on GPU with Tensor Cores by WMMA API

To show how your project organized the algorithm implementation, performance metrics and result verification

$ nvcc -o main main.cu -arch sm_75

Tensor Core is only supported by CUDA compute capability 7.0 and above

7.0 <=> Volta (Titian V / Quadro GV100)

7.5 <=> Turing (RTX 2080/ RTX 2080 Ti / Quadro RTX 6000)

• Programming Tensor Cores in CUDA 9
  • https://devblogs.nvidia.com/programming-tensor-cores-cuda-9/
• How to Implement Performance Metrics in CUDA C/C++
  • https://devblogs.nvidia.com/how-implement-performance-metrics-cuda-cc/
• NVIDIA Turing Architecture Whitepaper
  • https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf
• NVIDIA Volta Architecture Whitepaper
  • http://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf
• Tensorコアを使ってみた
  • http://proc-cpuinfo.fixstars.com/2018/10/tensorcore/
• CUTLASS: Fast Linear Algebra in CUDA C++
  • https://devblogs.nvidia.com/cutlass-linear-algebra-cuda/