Arch-delta Saves 80% Of Bandwidth On Upgrades(djugei.github.io)
djugei.github.io
Arch-delta Saves 80% Of Bandwidth On Upgrades
https://djugei.github.io/how-arch-delta-works/
4 comments
I wish there were a simple and easy way to P2P my pacman updates. I have several Arch machines/VMs that all need to download a gigabyte of updates at the same time.
This wastes not only my time and bandwidth, but adds extra load and bandwidth on the mirror. Why can't I have pacman sync with one of my other computers? Even if there's a different set of packages, the core OS is the same and can share packages with other machines.
I did look briefly into hosting a proper pacman mirror locally, but it seemed like way too much effort to save 10 minutes a month.
This wastes not only my time and bandwidth, but adds extra load and bandwidth on the mirror. Why can't I have pacman sync with one of my other computers? Even if there's a different set of packages, the core OS is the same and can share packages with other machines.
I did look briefly into hosting a proper pacman mirror locally, but it seemed like way too much effort to save 10 minutes a month.
Arch Wiki to the rescue. https://wiki.archlinux.org/title/Package_proxy_cache#
Currently using the dkarhttpd route with rsync. Everything is stored in a custom directory so that older version can be retained. Pacoloco looks interesting.
Currently using the dkarhttpd route with rsync. Everything is stored in a custom directory so that older version can be retained. Pacoloco looks interesting.
Just update one then rsync the package cache across? Or mount it over NFS for a hands-off solution. Or hack something up with inotify.
It always felt like such a waste to me how the DB always downloads tens of megabytes of data when likely only 1kB has changed. I mean I also really appreciate the beauty of how simple it is. But I'd bet even a delta against a monthly baseline file would reduce the data by >90%.
Also, it would be interesting to see how zstd --patch-from compares to the used delta library. That is very fast (as fast as normal zstd) and the code is already there within pacman.
For the recompression issue, there is some hard to find libraries that can do byte-exact reproducible decompression https://github.com/microsoft/preflate-rs but I don't know of any that work for zstd.