Kakao Brain announces torchgpipe, an implementation of GPipe in PyTorch as a handy library.
GPipe is a scalable pipeline parallelism library published by Google Brain. It leverages the training of a giant model which requires much memory. For instance, Google trained AmoebaNet-B with 557M parameters over GPipe.
0 comments
—
torchgpipe, a GPipe Implementation in PyTorch · HackerTrans
Kakao Brain announces torchgpipe, an implementation of GPipe in PyTorch as a handy library.
GPipe is a scalable pipeline parallelism library published by Google Brain. It leverages the training of a giant model which requires much memory. For instance, Google trained AmoebaNet-B with 557M parameters over GPipe.