Game Emulation via Neural Network(madebyoll.in)
madebyoll.in
Game Emulation via Neural Network
https://madebyoll.in/posts/game_emulation_via_dnn/
8 comments
Author here - thanks for the thoughtful points! I think we agree on a lot, but do disagree on the conclusion :)
My mental model (mostly == Karpathy's "software 2.0" one) is:
The interpretability advantage I'm claiming only applies to complicated programs: for sufficiently (and equivalently)-complicated programs, changing input->output behaviors specified explicitly in a training dataset feels much easier than trying to change the implicit input->output behavior of the corresponding zillion lines of source code.
--- Specific points:
> Most datasets have a reasonable amount of wrong input->output pairs
Definitely true! My counterpoint would be: most code contains a reasonable amount of bugs.
> it’s pretty easy for it to cheat
Hmm, I don't find that CycleGan article worrying: researchers trained a network to pass information through a constrained channel, and it succeeded (just not in the way they wanted). Quoting the article itself:
> this occurrence, far from illustrating some kind of malign intelligence inherent to AI, simply reveals a problem with computers that has existed since they were invented: they do exactly what you tell them to do.
> your dataset is likely to be much, much larger than the equivalent program.
I agree! But (again, for complicated programs) I am comfortable sweeping over a large dataset for errors (or asking a team of people to do so, or writing code to find errors, or even training tiny networks to find errors). I am not confident I could debug the equivalent zillion lines of source code just by reading it. The effect on input->output behavior from a single dataset example feels obvious, whereas the effect on input->output behavior from a single line of source code may depend on me understanding the other zillion lines...
> Right now we haven’t developed much tooling to help you map a sample bad input or behavior back to training data that might be relevant
Agreed (sadly)... though I think root-causing misbehaviors of complicated codebases is also painful.
> I don’t think it’s currently true that ML systems are simpler or more interpretable than a corresponding imperative program.
I agree for simple programs, but disagree for complicated ones; for sufficiently-complicated programs, I would much rather deal with the dataset representation.
My mental model (mostly == Karpathy's "software 2.0" one) is:
source code :: training data (and scoring function, if it's not simple MSE/MAE)
compiler :: training loop (and network arch :: compiler flags, maybe?)
compiled binary :: trained network (like the onnx file used in this demo)
The interpretability advantage I'm claiming only applies to complicated programs: for sufficiently (and equivalently)-complicated programs, changing input->output behaviors specified explicitly in a training dataset feels much easier than trying to change the implicit input->output behavior of the corresponding zillion lines of source code.
--- Specific points:
> Most datasets have a reasonable amount of wrong input->output pairs
Definitely true! My counterpoint would be: most code contains a reasonable amount of bugs.
> it’s pretty easy for it to cheat
Hmm, I don't find that CycleGan article worrying: researchers trained a network to pass information through a constrained channel, and it succeeded (just not in the way they wanted). Quoting the article itself:
> this occurrence, far from illustrating some kind of malign intelligence inherent to AI, simply reveals a problem with computers that has existed since they were invented: they do exactly what you tell them to do.
> your dataset is likely to be much, much larger than the equivalent program.
I agree! But (again, for complicated programs) I am comfortable sweeping over a large dataset for errors (or asking a team of people to do so, or writing code to find errors, or even training tiny networks to find errors). I am not confident I could debug the equivalent zillion lines of source code just by reading it. The effect on input->output behavior from a single dataset example feels obvious, whereas the effect on input->output behavior from a single line of source code may depend on me understanding the other zillion lines...
> Right now we haven’t developed much tooling to help you map a sample bad input or behavior back to training data that might be relevant
Agreed (sadly)... though I think root-causing misbehaviors of complicated codebases is also painful.
> I don’t think it’s currently true that ML systems are simpler or more interpretable than a corresponding imperative program.
I agree for simple programs, but disagree for complicated ones; for sufficiently-complicated programs, I would much rather deal with the dataset representation.
> The effect on input->output behavior from a single dataset example feels obvious, whereas the effect on input->output behavior from a single line of source code may depend on me understanding the other zillion lines...
Sure, but you have no idea how much weight any given sample contributes to the model. Maybe the behavior is included. Maybe not. The weight assigned could even vary drastically between runs with different random seeds!
I probably picked a bad paper with CycleGAN. My main point is that the mapping from data to behavior is non-trivial and there are lots of examples of this in the lit. Medical models that realize that the output variable is encoded in the text in the upper right of the image rather than attempting to analyze the image, etc.
Sure, but you have no idea how much weight any given sample contributes to the model. Maybe the behavior is included. Maybe not. The weight assigned could even vary drastically between runs with different random seeds!
I probably picked a bad paper with CycleGAN. My main point is that the mapping from data to behavior is non-trivial and there are lots of examples of this in the lit. Medical models that realize that the output variable is encoded in the text in the upper right of the image rather than attempting to analyze the image, etc.
The demo feels like a dream. You try to move towards things, but they transform and you can not reach them. You can also not go back, the place you came from totally changed. Suddenly, you're somewhere else completely.
Awesome idea and solid article!
Awesome idea and solid article!
Not enough is being done to preserve context across time and space.
Needs more recurrency?
Needs more recurrency?
From the author's twitter, here are links to a download of the network and to a site that lets you view the network architecture as a flow chart: https://twitter.com/madebyollin/status/1566886407117107200?s...
This was an interesting application of neural nets and a good write up! I’m curious how a game might be made from scratch using a neural net, i.e. what would it train on? I think this would be a cool technique to have at our disposal because games implemented as neural nets might make data mining harder, so all the cool secrets in the game wouldn’t be immediately exposed upon release :)
Don't forget to try the demo right in your browser: https://madebyoll.in/posts/game_emulation_via_dnn/demo/
The conclusion is pretty wrong though.
> Regardless of how complicated your program’s behavior is, if you write it as a neural network, the program remains interpretable. To know what your neural network actually does, just read the dataset.
This is not true at all? That’s like saying “all binaries are open source, just read the machine code”. Most datasets contain a reasonable amount of pollution (even the author ran into this). And if you let the model train on its own output it’s pretty easy for it to cheat: https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-fr...
Moreover current ML techniques are pretty sample inefficient. So your dataset is likely to be much, much larger than the equivalent program. Right now we haven’t developed much tooling to help you map a sample bad input or behavior back to training data that might be relevant. So “just reading the data” seems like lots more work than debugging a modern program.
I do think we’ll get better at debugging this stuff in time, but I don’t think it’s currently true that ML systems are simpler or more interpretable than a corresponding imperative program.