Lo and behold, a nice arithmetic coding implementation that wasn't written by an LLM! A sight for sore eyes – a treat, even. Looks like it was written by someone else though.
Haha. Unfortunately is my regular voice, since long before I started using Codex. You can check through some of my old writing. It definitely could've gotten worse though. Not sure if I'm training on Codex, or Codex is training on me...
This is certainly part of it! My point was that focusing on problems proposed by others is one very specific and pretty short-term mode of thinking. Good researchers improve benchmark scores. Great researchers think about what problem they're solving.
Incredible concept and a very well-crafted site. I scored very low, but then very high with my legal name. It seems DeepSeek knows a lot of arxiv papers (or at least, about the authors).
> The app does absolutely no work in the background. It works by simply existing as a running process, thanks to having the same bundle identifier as the Music app.
I love clever, low-or-no-code engineering solutions like this. You typically need to understand a systems very deeply to reach this level of elegance. In this case, one has to understand exactly what happens when the play button is pressed in Mac OS, how bundle identifiers work, etc. And the outcome is an app with almost no code at all – just a collision – it's beautiful.
(As an aside, coding agents are terrible at this kind of thing; I'd guess Codex as of right now would write some overpowered application that polls in a loop looking for Music App starts and killing them)
It's great that people are starting to take continual learning seriously, and it seems like Jessy has been thinking about LLMs and continual learning longer than almost anyone.
I especially like this taxonomy
> I think of continual learning as two subproblems:
> Generalization: given a piece of data (user feedback, a piece of experience, etc.), what update should we do to learn the “important bits” from that data?
> Forgetting/Integration: given a piece of data, how do we integrate it with what we already know?
My personal feeling is that generalization is a data issue: given a datapoint x, what are all the examples in the distribution of things that can be inferred from x? Maybe we can solve this with synthetic datagen. And forgetting might be solvable architecturally, e.g. with Cartridges (https://arxiv.org/abs/2506.06266) or something of that nature.