综合综述
https://journalofbigdata.springeropen.com/articles/10.1186/s40537-023-00829-x
Collective operation
Broadcast
Reduce
All-Reduce
Prefix-Sum/Scan
Barrier
Gather
All-Gather
Scatter
All-to-all
BLOG
Record, summarize, and improve.
2026-08-20
分布式深度学习