DualPipe: a bi-directional pipelined parallel algorithm to improve the efficiency of large-scale AI model training (DeepSeek Open Source Week Day 4)
General Introduction DualPipe is an open source technology developed by the DeepSeek-AI team focused on improving the efficiency of large-scale AI model training. It is an innovative bi-directional pipelined parallel algorithm primarily used in DeepSeek-V3 and R1...