Interesting. Does this mean larger models could be ran on less memory? It looks like it uses 15-20x less memory. Could a 671B DeepSeek R1 be ran in just ~40-50GB of memory? It sounds like it'd be 1/3 as fast though (<1tk/sec).
I created an offline version of YouTube (for Mac). I use Downie to download videos I'm interested in (usually videos download in 10-30 seconds). Then I go to the web app and click play. Using a shell command, it launches IINA and starts playing the video.
I have a mark watched and delete button so that YT's algorithm still works to surface interesting videos.
Having offline videos allows me to watch videos like I couldn't before (even on a stable 1Gbps connection). I can fast-forward instantaneously and there are no ads. Subtitles are also downloaded and a thumbnail.
Another nice feature inherent of offline video is if you want to grab your laptop and go somewhere (I'll sometimes go to the park and eat), you don't have to worry about your video stopping because you don't have internet/need to tether.
The whole app is contained in one executable file (you'll need Ruby) and stores its data in the JSON files that Downie provides.