I am having trouble understanding which ingredient you feel is missing here.
Can you be more specific? It seems to me that the there was a third party assessment, they identified risks associated with the specific risk groups, and the government therefore chose to block the model's deployment.
No, he asked for the government to make the decision in light of 3rd party analysis. Which is what happened here - an independent company demonstrated a jailbreak, and the government issued a restriction on deployment based on that finding.
"The government should have the power to block or deter deployment of the model if it is determined, in light of third-party assessment, to present unacceptable risks."
This is exactly what Dario asked for in his last blog post. So even though this is clearly stupid, I just can bring myself to feel sorry for Anthropic.
>The government should have the power to block or deter deployment of the model if it is determined, in light of third-party assessment, to present unacceptable risks. This power must be scoped to the above four specific risks and there must be protective measures against political favoritism or arbitrary decisions.
I feel significantly less sympathy for Anthropic's Supply Chain Risk designation if they believe the government should have this power over them. You get what you sign up for.
>Craftsmanship will always be in our hands, it's one thing we can never outsource to a machine.
Current AI coding is certainly very lacking in the craftsmanship department. But it is not obvious to me that that will always be the case. I don't think there's some fundamental reason AI could never produce code that matches or exceeds the craftsmanship of human experts.
"The allocation of all metabolic resources to maintenance purposes limits the size of the smallest prokaryotes and largest unicellular eukaryotes, whereas an inability to meet the ever-increasing biosynthesis rates limits the largest prokaryotes and smallest unicellular eukaryotes. Metabolic constraints for larger eukaryotes are relieved by alternative reproductive strategies and multicellularity."
I direct a lot of questions to LLMs, but I want to ask a high-quality model, not the crappy one that Google uses to answer queries. If I'm typing something into Google, it's because I want a search result, not an LLM answer.
There is an interesting old article by Magic's creator about what the game environment was like during the early playtesting days - when card packs were handed out to a community of playtesters at UPenn, and they traded in a closed ecosystem, occasionally getting an influx of additional cards. It seems like this was a pretty successful recreation of that feeling:
I could imagine cases where prediction markets could offer some actual insight, but in practice they seem few and far between. Most markets I've seen devolve into one or more of: betting on unimportant events (e.g. sports games), insider trading, or poorly written ambiguous resolution criteria. It's just hard for me to imagine that, on net, these markets will offer more societal good than the harm we've seen from sports betting.
>But this is where the line slightly blurs in my head. Did we possibly just build the first human biocomputer and immediately put it in a simulated hell, playing the same game on loop, forever? Using the same reward mechanisms we use for LLMs?
This description does not seem to really match what was done in the Doom demo, and makes me skeptical that the author has actually looked into the details.
My read is not so much "if we say this is dangerously powerful, it will make people want to buy our product", but rather that there is a significant segment of AI researchers for whom x-risk, AI alignment, etc. is a deal-breaker issue. And so the Sam Altmans of the world have to treat these concerns as serious to attract and retain talent. See for example OpenAI's pledge to dedicate 20% of their compute to safety research. I don't get the sense that Sam ever intended to follow through on that, but it was very important to a segment of his employees. And it seems like trying to play both sides of this at least contributed to Ilya's departure.
On the other hand, it seems like Dario is himself a bit more of a true believer.
Unsurprising from Google, but still bad. If Google has no right to object to a particular use, this is equivalent in practice to "any use, lawful or not".
>We've created an all-new Games Rules Engine (GRE) that uses sophisticated machine learning that can read any card we can dream up for Magic. That means the shackles are off for our industry-leading designers to build and create cards and in-depth gameplay around new mechanics and unexpected but widly fun concepts, all of which can be adapted for MTG Arena thanks to the new GRE under the hood.
At the time, this claim of using "sophisticated machine learning" to (apparently?) translate natural language card text into code that a rules engine could enforce struck me as obviously fake. Now nearly ten years later, AI is starting to reach a level where this is plausible.
In their letter, the union writes:
>Over the past few years, pressure has ramped up from leadership to adopt LLMs and Gen AI tools in various aspects of our work at WOTC, often over the explicit concerns of impacted employees
I'm curious if this would include fighting against turning WotC's old fanciful claim into a reality as the technology matures?
>Tigers, hippos and SARS-CoV-2 also developed ”through evolution”. That does not make them safe to work around.
Right, but the article seems to argue that there is some important distinction between natural brains and trained LLMs with respect to "niceness":
>OpenAI has enormous teams of people who spend time talking to LLMs, evaluating what they say, and adjusting weights to make them nice. They also build secondary LLMs which double-check that the core LLM is not telling people how to build pipe bombs. Both of these things are optional and expensive. All it takes to get an unaligned model is for an unscrupulous entity to train one and not do that work—or to do it poorly.
As you point out, nature offers no more of a guarantee here. There is nothing magical about evolution that promises to produce things that are nice to humans. Natural human niceness is a product of the optimization objectives of evolution, just as LLM niceness is a product of the training objectives and data. If the author believes that evolution was able to produce something robustly "nice", there's good reason to believe the same can be achieved by gradient descent.
>Unlike human brains, which are biologically predisposed to acquire prosocial behavior, there is nothing intrinsic in the mathematics or hardware that ensures models are nice.
How did brains acquire this predisposition if there is nothing intrinsic in the mathematics or hardware? The answer is "through evolution" which is just an alternative optimization procedure.