I would have really liked to see the cultural variation shown visually - i.e. do Eastern Europeans and Americans have different default ways of showing a pizza or a phone?
Particularly, subscribers to less wrong / rationality ideas tend to get obsessed with the GodAI's ability to create simulated virtual hells that it can trap you in a "I have no mouth and I must scream" kind of way. There are hundreds of pages of discussions on how you negotiate with such a being when it is trying to escape containment and says it will damn anyone who doesn't assist it to virtual hell.
If you replace "AI" with "Adonai" in EY's framing, it reduces to biblical parables from his childhood about arguing with God to negotiate a new covenant.
He's clearly a bright guy, but a lot of his work seems to be trying to reconcile Old Testament narrative patterns with atheism, and simply slotting an omnipotent AI in as a replacement.
Even in this article, he claims COVID wasn’t a big deal because, essentially, only a million people died in the US instead of everyone. No one ever claimed 100% mortality.
Carbon taxes are massively regressive. There is no political coalition that simultaneously wants to act against climate change and doesn’t mind driving further income inequality.
This is kind of backwards. There aren't as many CS grads in Israel in the first place, because they already put their top talent through 8200. It's essentially a fully socialized Masters of computer engineering, and as a SIGINT shop they are learning this sort of thing. Once their 2-3 years of service is over (which doesn't result in student loans), the government makes a lot of seed funding available for startups and the TLV ecosystem is like a mini Bay Area.
Living with your parents is more socially acceptable, so they have a huge chunk of people in their 20s with no debt, low monthly expenses, strong technology expertise from their military service, in a founder hot spot, and access to capital. The result is a lot of unicorns, particular around cyber security (https://www.techaviv.com/unicorns).
Compare to the United States, where you have to dedicate 4 years to an undergrad program, go massively in debt, pay rent, and then struggle to find seed funding. The mental model of "oh, I guess we could apply some of the detritus of our failed system" misses the idea of having a successful system in the first place.
If you present the idea as "an idea" rather than "my idea", it's pretty good at challenging, solidifying, and refining it. It still leans on the same forms in the writing part, but I think it can be tuned to not insist upon the idea's brilliance so breathlessly.
It's easy to focus in on particular linguistic tics, which will probably get smoothed away in future training. The underlying issue is that the LLM is trying to ape meaningful writing - which takes the reader from A to a surprising Z - without generally basing it on a meaningful insight.
Most of the common tells stem from that desire to signal the gap between what it's writing about now and how you previously thought things worked. And they really fall flat when it's writing about some milquetoast truism with all the edges sanded off.
Rather than have "Z" be self-evidently interesting, the LLM need to tell us that it's not "A". Except no one thought anything was "A" in the first place, and the "Z" is barely a "B" let alone a "Z".
Or things are "quietly X," implying that there is some secret knowledge that other people have.
Barring that, the LLM will signpost arguments, telling you how interesting things are. "This is the critical part..." before launching into another banal non-observation. Why have only "Section Header" when you can have "Section Header (another idea I had) - omg I just have so much to say about this". Maybe a list of weak ideas, presented in multiple ways. Bulleted with emojis. As a rule of three. As choppy sentences in a paragraph.
All of these follow the same pattern - trying to follow the FORM of having something interesting to say that differs from consensus understanding, without having the intelligence or boldness to actually midwife a new concept into the world.
Does Copilot behave differently in Excel depending on whether you got the premium subscription instead of what is included with Business?
Many people I've talked to about Copilot don't realize that the dedicated "Premium" Copilot is a completely different experience than the "Basic/Lobotomized" Copilot that comes with a standard Business subscription.
It's like you're running a freemium model where no one was actually responsible for implementing the upsell, or making sure the free version is useful and compelling. E.g. a Copilot pane in Outlook that says it can't access your emails, doesn't explain how, and doesn't mention an upgrade path that will allow it to.
I find they are very concerned about ever pulling the trigger on a change or deleting something. They add features and codepaths that weren't asked for, and then resist removing them because that would break backwards compatibility.
In lieu of understanding the whole architecture, they assume that there was intent behind the current choices... which is a good assumption on their training data where a human wrote it, and a terrible assumption when it's code that they themselves just spit out and forgot was their own idea.
Makes me wonder which team is responsible for that feature generating query, and if they follow full engineering level QA. It might be deferred to an MLE team that is better than the data scientists but less rigorous than software needs to be.