MCR-DL Features

MCR-DL v0.1 is an interface between PyTorch and communication backends such as MPI and NCCL that enables high-performance communication, simple communication extensibility, and a suite of PyTorch communication benchmarks.

ParaInfer-X Features

MPI4DL Features

Horovod with MVAPICH2 Features

MPI4cuML Features

OSU-Caffe 0.9 Features

OSU-Caffe derives from Caffe, which is a Deep Learning Framework that provides the flexibility to design and enhance DL models. All the features available with the NVIDIA's fork of the BVLC Caffe are available with this release. OSU-Caffe offers additional features and mechanisms that take advantage of the HPC resources. It is an MPI distributed version that scales-out on multi-GPU nodes. It takes advanatge of the optimized CUDA-Aware MPI to boost its performance on GPU Clusters. OSU-Caffe re-designs the DL workflow to provide overlap of the computation and communication. Further, it takes advantage of efficient large message MPI collective communication operations from GPU buffers that efficiently exploit GPUDirect RDMA, CUDA IPC, CUDA Kernels and Core-Direct features.

The list of features for supporting distributed and large scale DL frameworks.

RDMA-TensorFlow 0.9.1 Features