Would it be bad if we made advertising... illegal? In small part because it would reduce the principal-agent disparity between what you want to do and what Google wants you to do, but more importantly because it would remove the incentive to create blogspam and game the results.
Advertising serves a positive purpose by informing people about products that they will like and otherwise would not find. It also serves a negative purpose by attempting to cause people to make decisions which are not in their best interests.
I am genuinely curious what proportion of ads seen serve each of those two purposes.
That dip is more than 4M people, so I think it's still unexplained. Perhaps due to a shift in response rates or another artifact of COVID churn in the survey methodology?
Handling delays (and the uncertainty they entail) is a huge challenge, and I think it'll be a rich area of research. The simplest part of the problem is that delays in action or perception also slow the propagation of reward signals, and credit assignment is still a really hard problem.
Thinking further afield, future models could learn to adapt their expectations to fit the behavior of a particular opponent. This kind of metalearning is pretty much a wide open problem, though a pair of (roughly equivalent) papers in this direction recently came out from DeepMind: https://arxiv.org/abs/1611.05763 and OpenAI: https://arxiv.org/abs/1611.02779 It's going to be really exciting to see how these techniques scale.
Yup, it's definitely an advantage to get all the correct values from the game state. But not as much as you might think; the vision portion of a DQN or similar trains quite quickly.
Plus, our bot doesn't have any clue about projectiles. We don't know where they live in memory, so the network doesn't get to know about them at all.
I'd love to do live variable inspection (like the Chrome Dev Tools) — it's something I'm thinking about for the future.
There's a (nascent) project called `jmp` (for Jupyter Messaging Protocol) that implements kernel communication in JS: https://github.com/n-riesco/jmp. I'll probably contribute to that and factor out my kernel communication.
The development experience for statically typed languages is often _amazing_ compared to that of dynamically typed ones; we sometimes choose dynamically typed ones anyway because of the flexibility and speed of development they give. I'd be very happy with an environment that provided dynamically typed languages the same analysis, checking, completion, etc that statically typed languages take for granted.
Jupyter provides a spec for a ZeroMQ-based messaging API, which kernels in each language have implemented. Hydrogen acts as a client to those kernels the same way that the Jupyter frontend does, sending execution requests to the kernel and displaying responses.
I'd love to see live test results in Atom too! It would make it so much easier to be confident in your code's correctness.
I like to break my code up into little functions that are easy to test, so that flow works well for me.
Even more powerful, though, is autocomplete from the running environment. It means that (just like in the Chrome Dev Tools) when you start typing something, the autocomplete suggestions include all of the valid names that are defined. It really helps in navigating complex APIs like Torch: https://www.dropbox.com/s/e6hriw5quwct65a/Screenshot%202015-...
I've been thinking a lot about what makes for a good development experience, and how to properly "factorize" an IDE to provide a consistent experience across different languages. The monolithic IDE experience can be great (once you get it set up) if all your work is in a single language. But for people like me, having a confusingly different development environment for each project is a nightmare.
This is a tiny step toward the dream of having a consistent core editor but with powerful tools for each language that allow introspection into the state of the running machine.
Most of the credit goes to the Jupyter team and the people who have implemented the kernels for various languages. The Jupyter API spec is what has made this possible. I'm really looking forward to what can be done next (hint: https://github.com/jupyter/notebook/issues/99)
Advertising serves a positive purpose by informing people about products that they will like and otherwise would not find. It also serves a negative purpose by attempting to cause people to make decisions which are not in their best interests.
I am genuinely curious what proportion of ads seen serve each of those two purposes.