logged back in to HN to comment on this. looks really sick - i've been saying for a while that a surprising amount of LLM inference really comes down to repetition down a known path.
it's good to see others have seen this problem and are working to make things more efficient. I'm excited to see where this goes.
it's good to see others have seen this problem and are working to make things more efficient. I'm excited to see where this goes.