It is one person who controls it, at his discretion who gets to share in the utility. What your saying can also be true without such an ownership model, right?
We were alive in an interesting time where a clerical caste of society was needed to get the most out of capital. Once this is no longer the case it could be that we return to something feudal.
They are new proofs and for sure useful but as far as I’ve understood mainly interpolative. E.g. an LLM can “create” a poem about a purple chicken as it has datapoints for “purple” and “chicken”, so it can create something plausible inbetween.
Similarly, in my mind it can interpolate proofs by interpolating between data points for technique A and technique B. This is novel and brute-forcing proofs this way is useful. It is analogus to how sometimes it can generate programs that pass unit tests, I think.
However, creating fundamentally new concepts outside of the interpolated datapoints is not something I am convinced of. Maybe it can extrapolate some things, if correct add it as a data point, continue. Essentially a search, and it would be amazing if this works and maybe we can get some recursive improvement this way. But the “ideas” it will use to conduct this search are a function of the input data points as well, and thus in my view fundamentally limited in novelty. I am not discounting the usefulness, but I am not convinced you can just keep doing this indefinitely scaling intelligence exponentially.
Of course nobody can know yet really and I am just speculating just like you. But I also think the “experts” Sam and Dario also don’t know, and given their incentives I am not really convinced by them.
I think having a technical argument is important, as we live in a time with lots of hype merchants who stand to benefit from record breaking IPOs. Propaganda can affect us all, how do you know you aren’t being sold to?
So, what your saying is that there is a perhaps linear, perhaps exponential increase, and that you are projecting that increase forward indefinitely. Let me know if this is unfair.
Counter argument: does anything else work this way? E.g. Moores law had an end too right? I would argue that the core tech breakthrough (Transformer-based LLM) has been improved, but no fundamental further innovation seems to have been made. The current architecture fundamentally hallucinates, even Fabel even on trivial problems. I.e. as number tokens increase error likelihood goes to infinity. How then, can this scale recursively to infinity?
I just cant get over this term, do you honestly believe in this? I use AI daily and while it is super useful I see too many limitations for it to “recursively improve and cause an intelligence explosion”.
1. Clearly, the people selling AI with this idea benefit greatly with this promise of infinite upside. Can you can trust them?
2. Singularity essentially requires to handwave away a lot of baked-in issues with LLMs or rely on unrealized innovation.
Counterpoint: what if he lied in the prospectus? With Elons proven track record of misleading investors and customers I think that possibility has to be considered.
Sounds like a similar track, and I agree that its a useful skill and talent.
What I mainly noticed was, after really understanding my domain, the confidence of the SWEs I was working with despite being incorrect. Now I am a SWE and I try to stay humble.
I think more broadly that grabbing attention with predictions and hot takes has become lucrative, and we definitely don’t celebrate prediction accuracy.