I’m really happy to see this take. It’s not the first time but it’s not said often enough. I once had the thought that anything AI can do really well is probably something that should not be being done at all. That’s an overly broad statement but I think there’s some truth in it. The grand challenge of software engineering is to find beautifully elegant and precise ways to express what we want the computer to do for us. If we can find them, it will be better to express ourselves in those ways than to prompt an AI than do it for us, much in the same way that a blog written by an LLM is not worth reading.
This. I believe it’s the most important question in the world right now. I’ve been thinking long and hard about this from an entirely practical perspective and have surprised myself that the answer seems to be our capacity to love. The idea is easily dismissed as romantic but when I say I’m being practical I really mean it. I’m writing about it here https://giftcommunity.substack.com/
There's a better way but it requires a very large space like a big empty parking lot.
...and that's it! Turns out the hard part is not riding a bike but riding a bike in a straight line. Once you've got the hang of riding wherever the bike seems to want to go, you can gradually learn to get it under control. Surprisingly easy!
Came to the comments looking for this. The term alignment-faking implies that the AI has a “real” position. What does that even mean? I feel similarly about the term hallucination. All it does is hallucinate!
I think Alan Kay said it best - what we’ve done with these things is hacked our own language processing. Their behaviour has enough in common with something they are not, we can’t tell the difference.
Any recommendations for thinkers writing good analysis on the implications of superintelligence for society? Especially interested in positive takes that are well thought through. Are there any?
Ideally voices that don’t have a vested interest.
For example, give a superintelligence some money, tell it to start a company. Surely it’s going to quickly understand it needs to manipulate people to get them to do the things it wants, in the same way a kindergarten teacher has to “manipulate” the kids sometimes. Personally I can’t see how we’re not going to find ourselves in a power struggle with these things.
Does that make me an AI doomer party pooper? So far I haven’t found a coherent optimistic analysis. Just lots of very superficial “it will solve hard problems for us! Cure disease!”
It certainly could be that I haven’t looked hard enough. That’s why I’m asking.
Off topic but if I may. The way people use percentages to express multiples is confusing. A doubling is a 100% increase. So a 200% increase is 3x, and so on. Then at some point we forget about the +1 and 1000% is "10 times the sum it previously paid for software licenses". Just a pet peeve I guess : )
Interesting timing for me! Just a couple of days ago I discovered the work of biologist Olivier Hamant who has been raising exactly this issue. His main thesis is that very high performance (which he defines as efficacy towards a known goal plus efficiency) and very high robustness (the ability to withstand large fluctuations in the system) are physically incompatible. Examples abound in nature. Contrary to common perception evolution does not optimise for high performance but high robustness. Giving priority to performance may have made sense in a world of abundant resources, but we are now facing a very different period where instability is the norm. We must (and will be forced to) backtrack on performance in order to become robust. It’s the freshest and most interesting take on the poly-crisis that I’ve seen in a long time.
Very good point. For some types of problems maybe the answer is yes. For example porting. The reward function is testing it behaves the same in the new language as the old one. Tricky for apps with a gui but doesn't seem impossible.
The interesting kind of programming is the kind where I'm figuring out what I'm building as part of the process.
Maybe AI will soon be superhuman in all the situations where we know exactly what we want (win the game), but not in the areas we don't. I find that kind of cool.
https://www.kobo.com/gb/en/ebook/tracts-n-50-antidote-to-the...