Bellard has trained various models, so it may not be the specific 169M parameter LLM, but his Transformer-based `nncp` is indeed #1 on the "Large Text Compression Benchmark" [1], which correctly accounts for both the total size of compressed enwik9 + decompresser size (zipped).
There is no unfair advantage here. This was also achieved in the 2019-2021 period; it feels safe to say that Bellard could have likely pushed the frontier far further with modern compute/techniques.
Impressive numbers. Does anyone have any read or anecdotes on how much a small/mid/large company loses from low quality of software/bad practices (or conversely profits from the opposite)?
Seems like a challenging metric to measure, but always been curious on what the numbers look like.