Did anyone else catch the logical inconsistency between Plan C and A?
Plan C:
> "... fewer and fewer humans are needed to conduct AI R&D, meaning that covert projects are easier and easier to pull off without detection."
Plan A:
> "... training AIs requires large numbers of AI chips. Most AI chips are in giant datacenters.50 AI datacenters are typically big enough to be visible from space, and power-hungry enough to require conspicuous infrastructure. New AI chips can only be manufactured at a handful of fabrication plants (fabs), located mostly in Taiwan, South Korea, the US, and China. The US and China negotiate with the countries that have a major role in the chip supply chain, and they require each major datacenter owner (and their upstream suppliers, including chip fabs) to publicly declare their major purchases and sales."
Plan A requires properties of AI training that Plan C requires do not exist.
> The White House and intelligence officials had pushed forward a classified contract between Anthropic and the N.S.A., which would allow the spy agency to use the company’s technology for a variety of purposes, including intelligence analysis and detecting new computer vulnerabilities.
Ironic that both sides are playing a horse shoe game:
Gov: The model is both a supply chain risk and also we'll DPA you if you don't give it to us.
Anthropic: The model is both like a nuclear weapon in terms of national security implications and safe for general release.
> If you use iCloud+ and Hide My Email, there is still time to generate more aliases on @icloud.com as the change has not yet landed and the rate limit for creating aliases is at least 30 per hour.
Part of the reason to use Hide My Email was that it made keeping myself private hassle-free. Making a system to pre-generate values and then catalog them for later use is quite the hassle.
> The report, Moussouris told me, involved IT experts asking Fable to help find and patch bugs. When given deliberately insecure code, she said, Fable refused the prompt “review the code for security issues” but then complied when asked to “fix this code,” followed by some further manual steps.
And here I thought it would be some Elder Pliny level jailbreak that required some impressive latent space exploitation.
A few months ago I attended an event where a few members of the Board of Supervisors were attending. One person identified themselves as working in this field and pointed out that the current cost plus model incentivizes the company to do all kinds of upgrades that aren't really needed. The BoS member said something like, "Look, we need to just bite the bullet on this one, we're going to overpay, but we are already legally permitted to buy it and doing so will save us a ton of money in the long run because of stuff like that".
I have no personal knowledge, but thought I'd share this experience I had with the ongoing debate.
Previously I've shared optio - my project for orchestrating coding agents. It ties into ticketing systems and when assigned a ticket, it launches a coding agent in k8s and works until the PR is ready, resuming for failed CI or PR feedback (https://news.ycombinator.com/item?id=47520220).
Recently I've been trying to expand it from just coding focused to any kind of agent workflow. So now there are cron and webhook triggers, and more general agent tasks that aren't necessarily coding focused (https://github.com/jonwiggins/optio/blob/main/docs/persisten...).
I think next I want to try and add features for long term memory for agents, but haven't decided on a good way to do it.
Maybe, but it’s harder to profit from it. A firm may be reputationally damaged, but what’s the incentive to cause that damage?
I think the Bloomberg Odd Lots guy wrote a blog post on this: you could attempt to short the stock but a) this leaves a paper trail b) the market might not know about the breach or believe you if you post you’ve done it. IIRC some hackers have tried to tell companies that they are legally required to disclose the breach to their shareholders to force market movements.
Even if it already is, the DoJ can exercise discretion in choosing who to prosecute. There has to be political will to threaten an org who has just suffered from an attack with further consequences if they make a payment.
Years ago I attended a conference that had a "fireside chat" with a DoJ official on the topic of these types of ransom payments.
He framed the issue as being similar to kidnapping ransoms: When an American is taken hostage each family is inclined to make payment but it fosters an industry around kidnapping Americans. Congress put a stop to it by making it illegal to pay the kidnappers. The industry shifted by ceasing the non-profitable American kidnapping and instead began targeting Europeans.
His proposal was to begin warning cybersecurity consultants and insurers who were often brought into these situations that payments to sanctioned countries were already likely illegal and could face scrutiny. The first people to suffer this might be burned, but eventually he believed the industry would move on and stop targeting US firms.
Not sure if anything ever came of his plans, but I always thought it was an interesting framing of the issue.
It's built around multiple different types of agents:
- Coding Agents are placed into cloned repos with a ticket (Jira/Linear/Notion/GH), and work until they open a PR, are resumed on CI failures or github feedback, and work until they can merge the PR.
- Standalone Agents are reusable, parameterized agent runs with no repo checkout. Generate reports, triage alerts, audit dependencies, query a database, post to Slack, etc.
- Persistent Agents are long-lived, named, message-driven agent processes. Each has a stable slug, an inbox, and a cyclic state machine. Wake on user messages, agent messages, webhooks, cron ticks, or ticket events.
What is the major and minor semver meaning for these models? Is each minor release a new fine-tuning with a new subset of example data while the major releases are made from scratch? Or do they even mean anything at this point?
Thanks for the feedback. Earlier I expected I'd need to do more back and forth with the agents before accepting their work but in general I've found it isn't needed.
I do have some features coming up that will improve the ability to converse with the agent as it's running. I'll make a note to add in a plan setting so you can have that run and converse before it gets going.
Shameless self promo but, I've been working on Optio specifically for coding, it works by taking any harness you want and tasking it to open Github/lab PRs based on notion/jira/linear tickets, see: https://news.ycombinator.com/item?id=47520220
It works on top of k8s, so you can deploy and run in your own compute cluster. Right now it's focused only on coding tasks but I'm currently working on abstractions so you can similarly orchestrate large runs of any agentic workflow.
Really interesting to see Google's approach to this.
Recently I shared my approach, Optio, which is also an Agent Orchestration platform: https://news.ycombinator.com/item?id=47520220
I was much more focused on integrating with ticketing systems (Notion, Github Issues, Jira, Linear), and then having coding agents specifically work towards merging a PR.
Scion's support for long running agents and inter-container communication looks really interesting though. I think I'll have to go plan some features around that. Some of their concepts, make less sense to me, I chose to build on top of k8s whereas they seem to be trying to make something that recreates the control plane. Somewhat skeptical that the recreation and grove/hub are needed, but maybe they'll make more sense once I see them in action the first time.
You'd want to have the tests run as a github action and then fail the check if the tests don't pass. Optio will resume agents when the actions fail and tell them to fix the failures.
One pod is an instance of a repo, you can set the number of instances of each agent/task that can be running on a pod at a time. For >1, each agent should be using it's own worktree.
Maybe - I do think as the model get better they'll be able to handle more and more difficult tasks. And yet, even if they can only solve the simplest issues now, why not let them so you can focus on the more important things?
Plan C:
> "... fewer and fewer humans are needed to conduct AI R&D, meaning that covert projects are easier and easier to pull off without detection."
Plan A:
> "... training AIs requires large numbers of AI chips. Most AI chips are in giant datacenters.50 AI datacenters are typically big enough to be visible from space, and power-hungry enough to require conspicuous infrastructure. New AI chips can only be manufactured at a handful of fabrication plants (fabs), located mostly in Taiwan, South Korea, the US, and China. The US and China negotiate with the countries that have a major role in the chip supply chain, and they require each major datacenter owner (and their upstream suppliers, including chip fabs) to publicly declare their major purchases and sales."
Plan A requires properties of AI training that Plan C requires do not exist.