I feel you should be able to keep the high level idea in your head at once; of course not the details. I don't think I've ever completed a piece of code without having a scheme in my head, and where it should fit in it.
I've seen at least 6 such papers, all being like "<popular architecture> are actually <a bit older concept>". Neural networks are generic enough that you can make them equivalent to almost everything.
I can't get a read on this guy. I have strong bullshit feelings, but I'm not 100% sure.
1. He seems to have an awful lot of referral links - $60 for a bottle of olive oil?
2. He has not posted why is his regime like that - e.g. "Here is a paper about iron deficiencies, so I'm taking an iron supplement".
3. 90% of his website is "look at how dense my bones are", "look at how healthy my liver is" - bro I just wanted to see the actual regime, and you can't even post that.
None of those are victimless crimes. If you attempt to shoot someone and your rifle jams, you still exposed him to the risk of dying. This is a ridiculous strawman.
> Primarily, they are pure functions that accept a sequence of tokens and return the next token. The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Even if the function is capable of modeling certain aspects of chess.
I have two arguments against. One, you could argue that state is transferred between the layers. It may be inelegant for each chain of state transitions to be the same length, but it seems to work. Two, it may not have "states", but if the end result is the same, does it matter?