Concrete AI tasks for forecasting (2014)(aiimpacts.org)
aiimpacts.org
Concrete AI tasks for forecasting (2014)
http://aiimpacts.org/concrete-ai-tasks-for-forecasting/
43 comments
"Take a written passage and output a recording that can’t be distinguished from a voice actor, by an expert listener."
Role Playing Games would explode if this happened. The ability to generate speech on the fly would create dramatically more immersive worlds.
Role Playing Games would explode if this happened. The ability to generate speech on the fly would create dramatically more immersive worlds.
Why is this tagged 2014? The article is dated Dec. 14, 2016 and includes a reference to AlphaGo.
Because someone didn't understand the ISO-8601 date format.
2016-12-14 => 14 December 2016
2016-12-14 => 14 December 2016
Is dang reading this?
#22 - "system should write code that sorts a list, rather than just being able to sort lists."
Using symbolic techniques, synthesizing algorithms is tractable. See Sec5.3 on pg11- of http://saurabh-srivastava.com/pubs/popl10-synthesis.pdf - Figure 2(b) is the synthesized selection sort algorithm, and Table 3 has QuickSort synthesis time at 160seconds. Very old work (2010) and used templates to guide synthesis. SAT solvers have improved, and much more CPU cycles are available, so in 2016, exhaustively exploring the template space will be trivial.
I.e., rather than reconstructing code from concrete program execution traces, giving the synthesizer (AI?) the ability to manipulate symbolic spaces leads to a tractable search space. Finding specification-matching programs within this symbolic space is doable.
Using symbolic techniques, synthesizing algorithms is tractable. See Sec5.3 on pg11- of http://saurabh-srivastava.com/pubs/popl10-synthesis.pdf - Figure 2(b) is the synthesized selection sort algorithm, and Table 3 has QuickSort synthesis time at 160seconds. Very old work (2010) and used templates to guide synthesis. SAT solvers have improved, and much more CPU cycles are available, so in 2016, exhaustively exploring the template space will be trivial.
I.e., rather than reconstructing code from concrete program execution traces, giving the synthesizer (AI?) the ability to manipulate symbolic spaces leads to a tractable search space. Finding specification-matching programs within this symbolic space is doable.
Imagine the implications for the real world if you can solve #32. I wonder will it be able to detect emergent laws (e.g. thermodynamics, Newtonian gravity as an approximation to general relativity etc.)? Also it will be very interesting to see which experiments AI chooses to perform.
After we've achieved all these things, we can start asking when "general intelligence" will arrive.
Most humans are pretty bad at at least a few of those things. Also, some are tasks that the human brain is specifically optimized for - human natural language most likely plays well to lots of human-hardware-design-quircks. So using them a test sounds pretty bad / unfair - lots of human-level but independently evolved alien intelligences would probably fail miserably at them. You could have above-human intelligence barely reaching human performance at either of these tasks but vastly surpassing human intelligence at others...
Imho GI "will arrive" after the point that we have swarms of similar close-to-human-level ai-agents trying to model each other's behavior while competing and collaborating in artificial and natural environments.
All theories of self-consciousness that make some sense imho are about agents trying to model self-similar agents and then somewhow reframing this ability to model past and future self. So you could have self-conscious dog-level agents (yeah, I believe "self conscience" is quantitative and a dog can have some of it and a human a different level of it, not 0 or 1), or non-self conscious above-human-"general"-AIs (though I don't know if you'd want to label "general" any non-self conscious AI, and I think any non-self-conscious above-human AI would get "self conscience" quite fast if you were to push it be be "general purpose" - striving to solve more and more types of problems, you inevitably arrive to some formulation of the "model something like yourself in order to predict its actions" problem).
Imho GI "will arrive" after the point that we have swarms of similar close-to-human-level ai-agents trying to model each other's behavior while competing and collaborating in artificial and natural environments.
All theories of self-consciousness that make some sense imho are about agents trying to model self-similar agents and then somewhow reframing this ability to model past and future self. So you could have self-conscious dog-level agents (yeah, I believe "self conscience" is quantitative and a dog can have some of it and a human a different level of it, not 0 or 1), or non-self conscious above-human-"general"-AIs (though I don't know if you'd want to label "general" any non-self conscious AI, and I think any non-self-conscious above-human AI would get "self conscience" quite fast if you were to push it be be "general purpose" - striving to solve more and more types of problems, you inevitably arrive to some formulation of the "model something like yourself in order to predict its actions" problem).
Hardware plays a massive part too; I don't see a static micro processor becoming conscious anytime soon. Our brains though are literally physical neural networks, that grow, move, and change physically all the time.
It does not. Any kind of software can run on any kind of turing complete hardware. Unless you're one of those believing in the quantum mind theory https://en.wikipedia.org/wiki/Quantum_mind . Just as a human brain can simulate a microprocessor (although unusably slow and with errors), so can a microprocessor simulate a human brain (also unusable slow). Read a bit more on turing equivalence - a badly explained topic unfortunately, because you know how bad are mathematicians at explaining stuff, even if imho it should be part of junior school curricula in a simplified version. (Oh, and maybe it's brushed off because it can also lead to pretty trippy conclusions: take a cockroach brain which is likely turing complete, mount it as a cpu in some kind of computer, stuck in a maigc time-machine box that makes time run order of magnitude faster, add some really fancy input/output tech, and you could run a human mind on it... think of what the reincarnation weirdoes would milk from this :) )
I'm pretty sure that if you find a way to extract a mathematical model of a human's personality, and a way to transform it to reduce "chemical noise quircks" and other hardware dependent but unnecessary "features"/bugs, you could run it just fine on a big datacenter and have a working immortal human mind.
(I'm also pessimistic enough to believe that we'll solve the "general AI" problem before solving the "mind extraction" one unfortunately. Hopefully our creations don't destroy us completely before solving problem (2) and we have time to pass on some direct immortal legacy of what it means to be "human"...)
P.S. but yeah, I agree that engineering better suited hardware might be actually easier than getting the current type of hardware fast enough or than inventing the mathematical transforms needed to run a mind's set of equations on current type hardware... we're slowly getting to custom hardware for ai ( https://cloudplatform.googleblog.com/2016/05/Google-supercha... ), but I imagine the future will be more analog/clockless...
I'm pretty sure that if you find a way to extract a mathematical model of a human's personality, and a way to transform it to reduce "chemical noise quircks" and other hardware dependent but unnecessary "features"/bugs, you could run it just fine on a big datacenter and have a working immortal human mind.
(I'm also pessimistic enough to believe that we'll solve the "general AI" problem before solving the "mind extraction" one unfortunately. Hopefully our creations don't destroy us completely before solving problem (2) and we have time to pass on some direct immortal legacy of what it means to be "human"...)
P.S. but yeah, I agree that engineering better suited hardware might be actually easier than getting the current type of hardware fast enough or than inventing the mathematical transforms needed to run a mind's set of equations on current type hardware... we're slowly getting to custom hardware for ai ( https://cloudplatform.googleblog.com/2016/05/Google-supercha... ), but I imagine the future will be more analog/clockless...
Not saying you can't run any kind of software on turing complete hardware, but consciousness is far more that just software. The only proof of consciousness we have is that a brain is required, where software and hardware are the same thing. New thoughts are real physical connections in the brain.
If you need new hardware for new software (thought) then it's a pretty bad design.
Actually it is a superior design; hardware that can adapt to the needs of the software can optimise at will, enhancing the performance to ridiculously fast levels.
> but consciousness is far more that just software
Citation needed.
> The only proof of consciousness we have is that a brain is required
I think I get what you mean, but that's going a bit too far. For example, in 1950 we could have said "consciousness requires a planet within 100 miles", then the space race would have proved us wrong: being close to a planet isn't required, it's just a coincidence, likely due to life and evolution being more likely to spontaneously occur on planets than off planets.
I would say a similar thing about brains: the fact that up until now we've only seen consciousness in brains is a coincidence, likely due to complex behavioural circuitry being more likely to spontaneously occur in meat (brains) than rocks (chips).
Regarding computability, current scientific understanding goes something like the following:
- The universe, and everything inside it, seems to behave according to some sort of mathematical laws
- The known laws of physics are definitely incomplete (e.g. we have no quantum theory of gravity, we don't have enough experimental evidence to choose between theories of dark matter and dark energy, etc.)
- The known laws of physics seem to be a very good approximation in almost all 'non-extreme' circumstances (this is why we must build giant underground atom smashers if we're to find any discrepancies, and we still don't find any)
- The known laws of physics are computable, i.e. we can use a turing machine to implement the known laws of physics.
- Consciousness arises inside small, warm sacks of meat.
- Small, warm sacks of meat are a 'non-extreme' circumstance.
- Therefore, it seems highly likely that turing machines are sufficient to implement consciousness
This is a non-constructive argument, it doesn't give us an example of a conscious turing machine; yet it's not much different than, say, arguing that collections of atoms can form living creatures, without giving an exact definition of "life" or a receipe for constructing a living creature out of a collection of atoms.
This argument does make some assumptions, e.g. where I've said "seems to be" rather than "is"; these are areas where more research could help, but it seems unlikely that they'll yield refutations to the above argument. The burden of proof is on those arguing that consciousness is unlike anything we've ever studied.
Such a refutation would have to:
- Find new fundamental physics
- Show that this new physics is incomputable, unlike all previous theories
- Calculate a prediction from this new theory (quite tricky, since it's incomputable!), which contradicts predictions of existing theories
- Test the prediction experimentally
This would refute the "turing machines can implement known physics" part, but the argument might still be plausible regarding "non-extreme circumstances" (e.g. if your prediction involves black holes or other exotic situations).
To refute the argument convincingly, you'd need to demonstrate that your previously unknown effect can have macroscopic consequences at ambient temperatures, pressures, in a weak gravitational field, etc. and hence may occur in the brain.
Bonus points if you can demonstrate that your previously unknown, macroscopically-relevant, unlike-anything-else-in-science phenomenon has been routinely occuring inside pregnant women for millions of years (at least) without anyone noticing :)
Citation needed.
> The only proof of consciousness we have is that a brain is required
I think I get what you mean, but that's going a bit too far. For example, in 1950 we could have said "consciousness requires a planet within 100 miles", then the space race would have proved us wrong: being close to a planet isn't required, it's just a coincidence, likely due to life and evolution being more likely to spontaneously occur on planets than off planets.
I would say a similar thing about brains: the fact that up until now we've only seen consciousness in brains is a coincidence, likely due to complex behavioural circuitry being more likely to spontaneously occur in meat (brains) than rocks (chips).
Regarding computability, current scientific understanding goes something like the following:
- The universe, and everything inside it, seems to behave according to some sort of mathematical laws
- The known laws of physics are definitely incomplete (e.g. we have no quantum theory of gravity, we don't have enough experimental evidence to choose between theories of dark matter and dark energy, etc.)
- The known laws of physics seem to be a very good approximation in almost all 'non-extreme' circumstances (this is why we must build giant underground atom smashers if we're to find any discrepancies, and we still don't find any)
- The known laws of physics are computable, i.e. we can use a turing machine to implement the known laws of physics.
- Consciousness arises inside small, warm sacks of meat.
- Small, warm sacks of meat are a 'non-extreme' circumstance.
- Therefore, it seems highly likely that turing machines are sufficient to implement consciousness
This is a non-constructive argument, it doesn't give us an example of a conscious turing machine; yet it's not much different than, say, arguing that collections of atoms can form living creatures, without giving an exact definition of "life" or a receipe for constructing a living creature out of a collection of atoms.
This argument does make some assumptions, e.g. where I've said "seems to be" rather than "is"; these are areas where more research could help, but it seems unlikely that they'll yield refutations to the above argument. The burden of proof is on those arguing that consciousness is unlike anything we've ever studied.
Such a refutation would have to:
- Find new fundamental physics
- Show that this new physics is incomputable, unlike all previous theories
- Calculate a prediction from this new theory (quite tricky, since it's incomputable!), which contradicts predictions of existing theories
- Test the prediction experimentally
This would refute the "turing machines can implement known physics" part, but the argument might still be plausible regarding "non-extreme circumstances" (e.g. if your prediction involves black holes or other exotic situations).
To refute the argument convincingly, you'd need to demonstrate that your previously unknown effect can have macroscopic consequences at ambient temperatures, pressures, in a weak gravitational field, etc. and hence may occur in the brain.
Bonus points if you can demonstrate that your previously unknown, macroscopically-relevant, unlike-anything-else-in-science phenomenon has been routinely occuring inside pregnant women for millions of years (at least) without anyone noticing :)
Why do I need a citation? Our hardware is an approximation. Think of it like this: Over the course of history we have built smaller and smaller tools, from dumb tools such as hammers and anvils that we could mold visibly. Then later we have tiny chips that do nano scale computation, requiring other tools to even see what is going on and build them.
But evolution started from the smallest thing possible/in existence; Atom, quark, whatever, and built up into the single cell, then multi-cellular and so on. It also existed in a physical world where forces are constantly applied to everything it does. Opposed to a virtual world which is basically a vacuum.
Given a few billion years and this hardware is now known as the human body, built up of trillions of bacteria, undecipherable DNA and countless genes, protein folding and so on. So yes, there is a ton of stuff going on we don't understand. The amount of variables are incalculable.
To think we can replicate all this with some nano scale transistors, and mathematical approximations, is just bizarre. At best we can create an approximation of life, something that mimics it but isn't truly conscious.
Consciousness is as real as gravity, we can't definitely point to what they are, but they absolutely do exist.
But evolution started from the smallest thing possible/in existence; Atom, quark, whatever, and built up into the single cell, then multi-cellular and so on. It also existed in a physical world where forces are constantly applied to everything it does. Opposed to a virtual world which is basically a vacuum.
Given a few billion years and this hardware is now known as the human body, built up of trillions of bacteria, undecipherable DNA and countless genes, protein folding and so on. So yes, there is a ton of stuff going on we don't understand. The amount of variables are incalculable.
To think we can replicate all this with some nano scale transistors, and mathematical approximations, is just bizarre. At best we can create an approximation of life, something that mimics it but isn't truly conscious.
Consciousness is as real as gravity, we can't definitely point to what they are, but they absolutely do exist.
> It also existed in a physical world where forces are constantly applied to everything it does. Opposed to a virtual world which is basically a vacuum.
I don't understand this at all. I have no idea what is meant by "a virtual world which is basically a vacuum". As for "a physical world where forces are constantly applied to everything it does", how does this contradict anything about computability? Turing machines can compute worlds "where forces are constantly applied to everything"...
Are you talking about the sort of naive, approximate, numerical simulations used by scientists to model things like protein folding or galaxy formation? If so, those aren't actually implementing the known laws of physics, they're just convenient, somewhat-efficient approximations. By analogy, just because display machinery used by humans in the early 21st century just-so-happens to be based on a grid of pixels for convenience and efficiency, that doesn't preclude the existence of computations involving other representations, like vectors with unbounded precision (or, if you prefer, busy-beaver-bounded precision).
> there is a ton of stuff going on we don't understand.
This sounds like an argument from ignorance, which would be a fallacy in itself, but in this case we do understand some of what's going on! This "ton of stuff" is 'merely' a consequence of the underlying physics. Whether those consequences are complicated or not doesn't matter at all, because computability has nothing to do with being complicated.
If I run `while true; do python < /dev/urandom; done` my laptop will do "a ton of stuff" that I don't understand, some of which may be quite complicated; but I can say for certain that it will never produce jelly beans out of the USB port, since there is no mechanism for such a thing to occur, regardless of how complicated the signals in the wires are. On the other hand, if there were such a mechanism, it would only take one simple command (`sudo eject /dev/jellybean`) to cause its dispensement; no need for any appeal to complication at all.
Likewise, regardless of how complicated the interactions were in the formation of the Earth, life, our brains, etc. we know that such things must be computable, since there are no mechanisms in the known laws of physics which can lead to incomputability.
You can string together as many computable calculations as you like, far more than the universe could ever have performed in its entire history, and the result will still be computable. That's just how computability works.
In order to have anything that's incomputable, you need something other than computable components (i.e. jellybean-dispensing hardware, rather than more software). It doesn't even need to be complicated at all! If even just something tiny, like the magnetic moment of an electron, were incomputable by a Turing machine, given the laws of physics and history of the universe as input, that would be enough to support your argument. You don't need "tons of stuff", it's makes no difference. Yet even that seemingly tiny quirk would require that we throw away the known laws of physics. It seems unlikely.
> The amount of variables are incalculable.
Do you mean this as a linguistic gesture, or in a technical sense? For the latter, I would say it's maybe impractical to calculate, but certainly finite. We could upper-bound it by, say, Graham's number.
> To think we can replicate all this with some nano scale transistors, and mathematical approximations, is just bizarre.
I never said that, and as far as I can tell neither did any others in this thread.
Firstly, I would think of such claims as hubris. That such engineering feats could be achieved by a planetsworth of apes after only a few million years would be very surprising, and I certainly wouldn't want to bet on it happening either way before we become extinct.
Secondly, I don't understand what you mean by "all of this"; are you claiming that the entire history of the Earth must be reproduced before a conscious being can be made? In that case, what do you think is happening inside testicles, ovaries and wombs that can manage "all of this" in nine months?
Regardless, the arguments I made, and that of user nnq, were discussing computability. Computability (AKA universal computation) is a mathematical phenomenon regarding what can and cannot be computed by particular mathematical systems. The reason it's called universal computation is that, if something can be computed by one universal computer (e.g. the universe, a brain, a laptop, etc.) then it can be computed by all universal computers (e.g. the universe, a brain, a laptop, etc.).
This applies to all computations, whether we know what they are or not. Nobody (yet? ever?) knows what computation is being performed by some particular brain which gives rise to the phenomenon we call "consciousness", but we don't need to in order to know that that exact same computation can also be carried out by any other computer (e.g. a laptop).
No need for approximation. No need for appeals to ignorance. No need for appeals to "emergence". No need to even know what it is that brains do (since it must be a subset of what the laws of physics do, and the known laws of physics are computable).
But, as I said earlier, this isn't a constructive argument: it doesn't give us an x86 binary we can run. Also, as nnq pointed out, even if we did have such a binary, there's no reason to think that it might be in any way practical; it may take many trillions of times the age of the universe before it finishes calculating a single neuron activation. Such things don't matter at all for computability, but they're inconvenient for us mere mortals.
I don't understand this at all. I have no idea what is meant by "a virtual world which is basically a vacuum". As for "a physical world where forces are constantly applied to everything it does", how does this contradict anything about computability? Turing machines can compute worlds "where forces are constantly applied to everything"...
Are you talking about the sort of naive, approximate, numerical simulations used by scientists to model things like protein folding or galaxy formation? If so, those aren't actually implementing the known laws of physics, they're just convenient, somewhat-efficient approximations. By analogy, just because display machinery used by humans in the early 21st century just-so-happens to be based on a grid of pixels for convenience and efficiency, that doesn't preclude the existence of computations involving other representations, like vectors with unbounded precision (or, if you prefer, busy-beaver-bounded precision).
> there is a ton of stuff going on we don't understand.
This sounds like an argument from ignorance, which would be a fallacy in itself, but in this case we do understand some of what's going on! This "ton of stuff" is 'merely' a consequence of the underlying physics. Whether those consequences are complicated or not doesn't matter at all, because computability has nothing to do with being complicated.
If I run `while true; do python < /dev/urandom; done` my laptop will do "a ton of stuff" that I don't understand, some of which may be quite complicated; but I can say for certain that it will never produce jelly beans out of the USB port, since there is no mechanism for such a thing to occur, regardless of how complicated the signals in the wires are. On the other hand, if there were such a mechanism, it would only take one simple command (`sudo eject /dev/jellybean`) to cause its dispensement; no need for any appeal to complication at all.
Likewise, regardless of how complicated the interactions were in the formation of the Earth, life, our brains, etc. we know that such things must be computable, since there are no mechanisms in the known laws of physics which can lead to incomputability.
You can string together as many computable calculations as you like, far more than the universe could ever have performed in its entire history, and the result will still be computable. That's just how computability works.
In order to have anything that's incomputable, you need something other than computable components (i.e. jellybean-dispensing hardware, rather than more software). It doesn't even need to be complicated at all! If even just something tiny, like the magnetic moment of an electron, were incomputable by a Turing machine, given the laws of physics and history of the universe as input, that would be enough to support your argument. You don't need "tons of stuff", it's makes no difference. Yet even that seemingly tiny quirk would require that we throw away the known laws of physics. It seems unlikely.
> The amount of variables are incalculable.
Do you mean this as a linguistic gesture, or in a technical sense? For the latter, I would say it's maybe impractical to calculate, but certainly finite. We could upper-bound it by, say, Graham's number.
> To think we can replicate all this with some nano scale transistors, and mathematical approximations, is just bizarre.
I never said that, and as far as I can tell neither did any others in this thread.
Firstly, I would think of such claims as hubris. That such engineering feats could be achieved by a planetsworth of apes after only a few million years would be very surprising, and I certainly wouldn't want to bet on it happening either way before we become extinct.
Secondly, I don't understand what you mean by "all of this"; are you claiming that the entire history of the Earth must be reproduced before a conscious being can be made? In that case, what do you think is happening inside testicles, ovaries and wombs that can manage "all of this" in nine months?
Regardless, the arguments I made, and that of user nnq, were discussing computability. Computability (AKA universal computation) is a mathematical phenomenon regarding what can and cannot be computed by particular mathematical systems. The reason it's called universal computation is that, if something can be computed by one universal computer (e.g. the universe, a brain, a laptop, etc.) then it can be computed by all universal computers (e.g. the universe, a brain, a laptop, etc.).
This applies to all computations, whether we know what they are or not. Nobody (yet? ever?) knows what computation is being performed by some particular brain which gives rise to the phenomenon we call "consciousness", but we don't need to in order to know that that exact same computation can also be carried out by any other computer (e.g. a laptop).
No need for approximation. No need for appeals to ignorance. No need for appeals to "emergence". No need to even know what it is that brains do (since it must be a subset of what the laws of physics do, and the known laws of physics are computable).
But, as I said earlier, this isn't a constructive argument: it doesn't give us an x86 binary we can run. Also, as nnq pointed out, even if we did have such a binary, there's no reason to think that it might be in any way practical; it may take many trillions of times the age of the universe before it finishes calculating a single neuron activation. Such things don't matter at all for computability, but they're inconvenient for us mere mortals.
I think several of these will be post general intelligence. Especially the one(s) that require a simple explanation of reasoning, the generalized learning from an example or two and contextual understanding of natural language.
Do you mean self consciousness machines?
AGI is a pretty hard to define term, but I never saw it defined as "self-conscious". Most definitions are along the lines of "do what a human does", i.e. walk into a random house and make coffee or get a college degree, or, more mathematically, "Average a good score over all possible problems, weighted by their likelyhood and the compactness of your code"
"relatively well specified"
If these are "well specified", then the author and my boss are operating on some definition unknown to me.
Also, quite a few tasks don't seem to be obviously about AI. Stuff like:
> Play poker well enough to win the World Series of Poker.
It's one thing to play Starcraft or Go well. But implementing this is either mostly a question of luck, or an incredibly lifelike robot with incredible facial recognition skills. Very different tasks!
> Beat the fastest human runners in a 5 kilometer race through city streets using a bipedal robot body.
This requires no AI, just a lot of improvement in energy density and actuator technology.
If these are "well specified", then the author and my boss are operating on some definition unknown to me.
Also, quite a few tasks don't seem to be obviously about AI. Stuff like:
> Play poker well enough to win the World Series of Poker.
It's one thing to play Starcraft or Go well. But implementing this is either mostly a question of luck, or an incredibly lifelike robot with incredible facial recognition skills. Very different tasks!
> Beat the fastest human runners in a 5 kilometer race through city streets using a bipedal robot body.
This requires no AI, just a lot of improvement in energy density and actuator technology.
"It's one thing to play Starcraft or Go well. But implementing this is either mostly a question of luck, or an incredibly lifelike robot with incredible facial recognition skills."
In terms of strategy though, it is the next level up I'd say. Videogames come first, then more difficult games such as Poker come later where you need to be able to tell a bluff from the truth. Much more subtle.
"This requires no AI, just a lot of improvement in energy density and actuator technology."
It did say city streets though, which makes me think it would need to strategise and be able to:
1. Adjust speed and take corners close such as what you see in F1 races.
2. Switch between endurance and sprinting when necessary to maintain energy for the whole race.
3. Run behind others to stay in the tailwind and reduce energy cost.
All require human level sensory input and a strategising component.
In terms of strategy though, it is the next level up I'd say. Videogames come first, then more difficult games such as Poker come later where you need to be able to tell a bluff from the truth. Much more subtle.
"This requires no AI, just a lot of improvement in energy density and actuator technology."
It did say city streets though, which makes me think it would need to strategise and be able to:
1. Adjust speed and take corners close such as what you see in F1 races.
2. Switch between endurance and sprinting when necessary to maintain energy for the whole race.
3. Run behind others to stay in the tailwind and reduce energy cost.
All require human level sensory input and a strategising component.
I understood the running task as being through actual (i.e. populated) city streets; either a real city or a mockup built out of harm's way.
This would require real-time object detection and avoidance, following signs/social conventions/etc., complex route-planning, avoid people/cars/etc., predicting the movements of others, predicting how others will predict your predictions, etc.
This would require real-time object detection and avoidance, following signs/social conventions/etc., complex route-planning, avoid people/cars/etc., predicting the movements of others, predicting how others will predict your predictions, etc.
At human 5k speeds, on asphalt, taking corners close doesn't require speed adjustment. Traction is not really a problem. Just take a direct route.
Drag is quadratic with respect to speed, so switching between endurance and sprinting is sub-optimal. Just maintain pace.
And at these speeds and the required clearances for safety, the benefit from drafting is less than 5%. This is huge for runners pushing the limits of human performance, but if a robot has the same limits where this matters, it will probably be by design.
Drag is quadratic with respect to speed, so switching between endurance and sprinting is sub-optimal. Just maintain pace.
And at these speeds and the required clearances for safety, the benefit from drafting is less than 5%. This is huge for runners pushing the limits of human performance, but if a robot has the same limits where this matters, it will probably be by design.
Some of these are pretty awesome/crazy. I hope to see many achieved in the near future.
Yes and to be honest I wouldn't go so far as to call a lot of these "Concrete" tasks.
But it is an interesting and throught provoking list. And makes me consider more quantitive metrics for qualatitive things like how good a piece of music is.
But it is an interesting and throught provoking list. And makes me consider more quantitive metrics for qualatitive things like how good a piece of music is.
Number 31 ("Play poker well enough to win the World Series of Poker.") is a pretty empty statement, as 1 tournament does not reflect your skill at poker at all. Short term variance is too big of a factor when it comes to one 1 tournament result.
Well, in 2016 there were 6737 entrants [0], so I suppose the AI needs to have all its stars aligned or some other connection to god to win it without being good.
Biggest problems are usually not winning heads-up but meaningfully sampling the state space with more than a handful of players + accounting for the "no limit" part.
[0] https://en.wikipedia.org/wiki/World_Series_of_Poker#Main_Eve...
Biggest problems are usually not winning heads-up but meaningfully sampling the state space with more than a handful of players + accounting for the "no limit" part.
[0] https://en.wikipedia.org/wiki/World_Series_of_Poker#Main_Eve...
> Translate a text written in a newly discovered language into English as well as a team of human experts, using a single other document in both languages (like a Rosetta stone).
Huh? We can't even translate texts correctly between mainstream languages yet.
Huh? We can't even translate texts correctly between mainstream languages yet.
There is some interesting work applying word2vec on the voynich manuscript. It found all sorts of patterns that humans weren't aware of. It's theoretically possible even without strong AI.
It's a bit confusing, but I don't think this list is supposed to be a list of tasks we can do now. I think it's a list of well-defined (ish) tasks that are milestones to look out for in the future.
Or something like that anyway. It's not well explained.
Or something like that anyway. It's not well explained.
I'd really love to see -
'Beat tier one teams at least 50% of the time in Dota 2 across multiple patches'
It's cool that people are starting to work on Starcraft 2 AIs but the game is kind of dead so competition at the top isn't going to be as strong as it is in more popular games. Also I think Dota presents a much more interesting and more difficult challenge for AI makers.
'Beat tier one teams at least 50% of the time in Dota 2 across multiple patches'
It's cool that people are starting to work on Starcraft 2 AIs but the game is kind of dead so competition at the top isn't going to be as strong as it is in more popular games. Also I think Dota presents a much more interesting and more difficult challenge for AI makers.
The challenge is different: you've got way more units to control in SC2 than in Dota. And rushes can be punitive, which do not occur as frequently in Dota
The more units thing is part of the reason I think Dota is a better challenge, I can't help but feel a SC2 AI could use inhuman micro as a crutch whereas in Dota mechanical / micro skill doesn't provide quite as big an advantage.
Not too sure what you mean by 'rushes can be punitive' but Dota certainly has a variety of early game aggression strats which tend to be high risk / reward and are seen fairly regularly.
Not too sure what you mean by 'rushes can be punitive' but Dota certainly has a variety of early game aggression strats which tend to be high risk / reward and are seen fairly regularly.
IMHO the wording of point 12 is a silly way to pretend that Alphago has not completely achieved its goal. Human players learn Go using manuals that compile the experiences of millions of games. The fact that Alphago can use directly huge collections of games is not very relevant for its success.
There's a difference between needing 100,000,000 games and 50,000 for training to be the best. That difference is learning efficiency and astuteness - a good way of judging AI progress, no?
I think "50,000 games played" puts it a bit too simply. How many games has he watched how many equivalents of a game has he visualized as sequences of moves? If "played" is over 50k I'd be willing to bet "considered" is over 1,000,000.
We know that AlphaGo uses a Monte Carlo tree search, and presumably contains innovations and refinements to techniques applicable to turn based board games with perfect information and a clear binary win condition.
We also know it uses a Deep Learning algorithm to imitate play from the best humans. And presumably, being stronger than any human now, it could also recursively refine and train on its own games.
Do we know how important the Deep Learning component is? Would AlphaGo be just as strong or nearly so without that part?
I know that over the last few years chess engines have become dramatically more powerful, even on the same hardware, to the tune of hundreds of Elo points. They still stick to the classical techniques though (Alpha-Beta pruning, minimax is still the core). This is not to understate the refinements, but the general ideas are (I think?) only applicable to turn based games with finite moves and a binary mathematical criterion for victory.
Is it possible that the improvements AlphaGo has made are mostly of this type, and the Deep Learning parts are not that important to its actual play strength? Or can we totally rule this out? Is there an in depth discussion of this question somewhere?
We also know it uses a Deep Learning algorithm to imitate play from the best humans. And presumably, being stronger than any human now, it could also recursively refine and train on its own games.
Do we know how important the Deep Learning component is? Would AlphaGo be just as strong or nearly so without that part?
I know that over the last few years chess engines have become dramatically more powerful, even on the same hardware, to the tune of hundreds of Elo points. They still stick to the classical techniques though (Alpha-Beta pruning, minimax is still the core). This is not to understate the refinements, but the general ideas are (I think?) only applicable to turn based games with finite moves and a binary mathematical criterion for victory.
Is it possible that the improvements AlphaGo has made are mostly of this type, and the Deep Learning parts are not that important to its actual play strength? Or can we totally rule this out? Is there an in depth discussion of this question somewhere?
BTW, the point 32 is very inspiring. Could an AI imagine experiences to create models to explain the physic of our world. An AI to reinvent QM and relativity from scratch ? Maybe reconcile both ?
> The fact that Alphago can use directly huge collections of games is not very relevant for its success.
On the contrary, it is, and the _need_ for such quantities of data is one of the biggest challenges in neural networks.
How about 50,000 games + access to manuals ;)
On the contrary, it is, and the _need_ for such quantities of data is one of the biggest challenges in neural networks.
How about 50,000 games + access to manuals ;)
I see some criticism of the poker tasks, but that is unfounded. Poker is an important area of AI research because of bluffing (detection and when to do it). CMU is probably the most active in this area. See https://www.cmu.edu/news/stories/archives/2016/february/poke... and http://www.cs.cmu.edu/~noamb/poker.html. Their bot (Claudico) was demoed at NIPS15.
Interestingly, I've seen solid progress on 3, 6, and 8 in the last 2 months of 2016 and on 7 in 2017 already.