Top New Trends Comments Past Ask Show Jobs

Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism(github.com)

4 points·by yzh·2 anni fa·2 comments

Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism

https://github.com/sail-sg/zero-bubble-pipeline-parallelism

2 comments

ufotalent·2 anni fa

An interesting playground to understand the scheduler is here https://huggingface.co/spaces/sail/zero-bubble-pipeline-para...

QPHutu·2 anni fa

Cool!

Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism · HackerTrans