- Large binary files tend to be not very "deflatable"
- xdelta (used in Git to diff files) tries to load the entire content of a file into memory, at once.
This is why there are solutions like Git-LFS, where you keep your versions on a remote server / cloud storage and you use git to track only the "metadata" files.
DVC implemented its own solution, in order to be SCM agnostic and cloud flexible (supporting different remote storages).
- Large binary files tend to be not very "deflatable"
- xdelta (used in Git to diff files) tries to load the entire content of a file into memory, at once.
This is why there are solutions like Git-LFS, where you keep your versions on a remote server / cloud storage and you use git to track only the "metadata" files.
DVC implemented its own solution, in order to be SCM agnostic and cloud flexible (supporting different remote storages).
Here's more info comparing DVC to similar/related technologies: https://dvc.org/doc/dvc-philosophy/related-technologies
EDIT: formatting