HackerTrans
TopNewTrendsCommentsPastAskShowJobs

glinscott

no profile record

Submissions

Modern LZ Compression Part 2: FSE and Arithmetic Coding

glinscott.github.io
5 points·by glinscott·3 mesi fa·1 comments

CPU Only LLM (Vicuna-13B) with Web Search

garylinscott.substack.com
1 points·by glinscott·3 anni fa·0 comments

CPU Only LLM (Vicuna-13B) with Web Search

garylinscott.substack.com
4 points·by glinscott·3 anni fa·0 comments

comments

glinscott
·3 mesi fa·discuss
This is the second article in a series discussing modern compression techniques. The first one covered Huffman + LZ. This one covers optimal entropy coders (FSE and Arithmetic), and some additional tricks to get closer to the state of the art.

The full compressor and decompressor are just over 1500 lines of pretty compact C++: https://github.com/glinscott/linzip2/blob/master/main.cc.

It's been seven years since the first article! Hopefully not so long before the third (and probably final one).
glinscott
·3 anni fa·discuss
Btw, some great books on Steam locomotives/engines:

https://www.amazon.com/gp/product/B07NQ9JG2M - American steam locomotives 1880–1960. The author of drove locomotives, and was the transportation curator at the Smithsonian Museum. He wrote it over 30 years, and you can tell the amount of care and detail he put into with the details and history.

https://www.amazon.com/gp/product/B003UYU1PC - How Steam Engines Really Work, provides a modern perspective on the important systems and concepts for steam engines.

https://www.amazon.com/gp/product/B072BFJB3Z - The Perfectionists: How Precision Engineers Created the Modern World. Talks about how precision engineering was critical to the invention and wider use of steam engines.
glinscott
·3 anni fa·discuss
That's great. There is also this effort in New Zealand: https://mackwelloco.com/, focusing on "Modern Steam" techniques, with a small engine, targeting areas without easy access to diesel.

A good overview of modern steam techniques is here: http://advanced-steam.org/5at/principles-of-modern-steam/, where they focus on dramatically more efficient and clean burning engines. A fascinating rabbit hole to go down :).
glinscott
·3 anni fa·discuss
Steam engines are fascinating pieces of technology. The page for the trust itself is full of interesting engineering details: https://www.a1steam.com/.

Their next project is to build Britain's most powerful steam locomotive, things are making good progress! Some high level details here: https://www.a1steam.com/prince-of-wales/about-prince-of-wale..., with a bunch more detailed updates on the build here: https://www.a1steam.com/prince-of-wales/about-prince-of-wale.... Started in 2014!
glinscott
·6 anni fa·discuss
If anyone wants to experiment with training these nets, it's a great way to get exposed to a nice mix of chess and machine learning.

There are two trainers currently, the original one, which runs on CPU: https://github.com/nodchip/Stockfish, and a pytorch one which runs on GPU: https://github.com/glinscott/nnue-pytorch.

The SF Discord is where all of the discussion/development is happening: https://discord.gg/KGfhSJd.

Right now there is a lot of experimentation to try adjusting the network architecture. The current leading approach is a much larger net which takes in attack information per square (eg. is this piece attacked by more pieces than it's defended by?). That network is a little slower, but the additional information seems to be enough to be stronger than the current architecture.

Btw, the original Shogi developers really did something amazing. The nodchip trainer is all custom code, and trains extremely strong nets. There are all sorts of subtle tricks embedded in there as well that led to stronger nets. Not to mention, getting the quantization (float32 -> int16/int8) working gracefully is a huge challenge.