Biological systems climb up energy gradients and outcompete other systems.
Artificial systems should be able to climb given a suitable gradient. I think the hard part of AGI is going to be designing the environment and gradient to produce "intention", I don't think the hard part is studying the human mind to find out the "secret of intelligence"
The goal of AGI isn't silicon minds isomorphic to human minds at each level of interpretation. Just the existence of an intelligent system.
Reading the AI go FOOM debate solidified a lot of the mushy parts of my "singularitianism"
I think the linchpin of my belief is recursive self improvement. I think machine intelligences are a different kind of substance with different dynamics than the ones we typically encounter.
I don't think someone will compile the first AGI and presto there is it. I think a long running system of processes will interact and rewrite its own code to produce something, which eventually a reasonable boundary could be drawn to distinguish the system and anyone interacting with the system would say: "this thing is intelligent, the most intelligent thing on the planet". It would have instant access to all written knowledge, essentially unbounded power to compute new facts and information and model the world to as accurate of an approximation as needed to produce high confidence utterances.
I just don't see how a system like that couldn't come into existence one day. Issues around timelines are completely unknowable to me. But I would put a distribution of something like I would be surprised if it happened in the next 50 years and shocked if it didn't happen within the next 1000. Very fuzzy, but it "feels" inevitable.
If a collection of unthinking cells can coordinate and produce the feeling of conscious experience then I can't see what would stop silicon from producing similar behavior without many bounds inherent in biological systems.
Artificial systems should be able to climb given a suitable gradient. I think the hard part of AGI is going to be designing the environment and gradient to produce "intention", I don't think the hard part is studying the human mind to find out the "secret of intelligence"
The goal of AGI isn't silicon minds isomorphic to human minds at each level of interpretation. Just the existence of an intelligent system.