MSCCL++ v0.4.0 Release Plan (Released)

[chhwang]

Timeline

Released Date: Nov. 24th, 2023

Work Items

Communication Features and Interfaces

Platform support

1. • ROCm: add ROCm support (ROCm support #213)

GPU-side communication interfaces

1. • Interfaces: add tile-based communication interfaces (Add tile-based API #145)

Host-side interfaces

1. • Connection: async connection setup (Make setup async to get rid of setup() #167)
2. • SmChannel: improve constructors ([feature] sm_channel constructors need to be improved. #148)
3. • Python: Python binding for FIFO ([bug] the fifo from proxy cannot be bound by nb to python #153)
4. • Python: Python binding for DeviceSyncer ([feature] python binding for DeviceSyncer #156)
5. • ProxyChannel: warning ahead of IB CQ being full (Warning ahead of CQ being full #202)

Transports support

1. • IB: add IB gather (IB gather WIP #172)

Development Pipeline

1. • CI: add code coverage (Add code coverage report #208)
2. • Docs: update performance numbers on readme (Update documentation #217)
3. • Python Benchmark: add Python-based mscclpp-test ([Feature] Support python-based mscclpp-test #187, mscclpp-test in Python #204)
4. • Python Benchmark: robust correctness test (Robust correctness test #221)
5. • Docker: update Dockerfiles (Templatize Dockerfiles & update workflows #223)

Bug Fix

1. • Semaphore: use atomic (atomic for semaphores instead of fences #188)
2. • Python: GIL release (Release GIL for Python APIs with wait #190)
3. • CMake: check nvidia_peer module during build (Explicit build dependency on nvidia_peermem #201)
4. • mscclpp-test: AllReduce bug fix (Fix allreduce bug #197, Fixing the bug in allreduce1 #220)
5. • Interfaces: allow infinite waiting (Allow infinite waiting #200)
6. • Interfaces: fix a DeviceSyncer bug (Fix DeviceSyncer #222)

[chhwang]

Released: https://github.com/microsoft/mscclpp/releases/tag/v0.4.0