Spent a couple years essentially building exactly that at BlueRock. We had a partially formally verified microkernel that we would insert between the hardware and OS to intercept system calls. Under 10,000 lines of code by design, deployed in some very high-security environments.
Funnily enough we had to pivot to securing agentic AI (with a few pit stops along the way) because it was an extremely hard sell. We needed access to bare metal and the pitch to teams to allow us to do that was excedingly hard. Now, we've pretty much come full circle.
I believe AI could push people to look at securing Linux structurally but its hard for me to imagine they overcome the inertia we faced. Does bring up other interesting questions about AI-assisted formal verification, which even saying sounds weird. If you can verify a narrow slice of the call chain cheaply, you don't need to solve the whole kernel.
One more thing in regard to AI, the deployment model is shifting. Running agents on laptops is increasingly the wrong pattern. As they move to server infrastructure with real credentials and network access, the relevant security question changes too. Perimeter controls don't follow the workload, which is what's needed. The kernel approach addresses that but so does living/enforcing inside the execution itself.
Agreed on the floor being raised. The part I'd push back on is "fixing issues is now basically free." That's true for the issues that surface in code review or a failing test. The new class of issue is good-looking code that does something unexpected at runtime, usually through chains of tool calls that each looked fine in isolation. Those don't fail your tests. They fail in prod, sometimes quietly.
This is great! Love the idea. You should send to some art programs, sure they would get a kick out of it. Also gives another use case outside of just digital design.
Fair, had someone at a conference mention to me that he's working on crating agents with "beliefs". Sounds incredibly similar and quite frankly very spooky
Someone mentioned this previously in thread about how piracy (at least for sporting events) are a price issue. If they didn't charge an arm and a leg to watch (thinking of the NBA/NFL tv packages) they wouldn't have a problem.
I feel like this is always the case with new technology. People had the same reaction to the invention of the printing press. New is scary. It doesn't mean there aren't valid concerns, but unfortunately this feels a bit like an inevitability. The focus shouldn't be on stopping it, but how to maximize the gains and minimize the losses to the local communities where these are being built.
I weirdly feel like this is a newer issue. Hadn't had a problem running queries/actions previously up until this past month where it seems I'm constantly get hit with rate limits while not increasing my usage
Isolation over constraints sounds like the right philosophy. Containers give you a boundary but not vis into what ran inside them. Curious how much execution context Scion surfaces, w/o that you're still in a position similar to the LiteLLM attack where something can run and cause damage before you know it happened.
Ton of people have mentioned this but what you're doing with memory forking is pretty unique. Most sandboxes seem to just fork the filesytem and call it a day. Forking full VM memory mid-exec is taking it to another level entirely. Would be very interested to hear how the implementation looks under the hood, specifically how you handle dirty memory pages across forks without the pause ballooning.
Had not even thought about having kids in the house with an AI robot. Obviously there are the more apparent issues (asking about things they don't need to know) but saw a recent article about how ChatGPT and some other models have an implicit confirmation bias when interacting with users. I don't need my kid to reinforce bad ideas from their cute, all knowing, robot friend.
Funnily enough we had to pivot to securing agentic AI (with a few pit stops along the way) because it was an extremely hard sell. We needed access to bare metal and the pitch to teams to allow us to do that was excedingly hard. Now, we've pretty much come full circle.
I believe AI could push people to look at securing Linux structurally but its hard for me to imagine they overcome the inertia we faced. Does bring up other interesting questions about AI-assisted formal verification, which even saying sounds weird. If you can verify a narrow slice of the call chain cheaply, you don't need to solve the whole kernel.
One more thing in regard to AI, the deployment model is shifting. Running agents on laptops is increasingly the wrong pattern. As they move to server infrastructure with real credentials and network access, the relevant security question changes too. Perimeter controls don't follow the workload, which is what's needed. The kernel approach addresses that but so does living/enforcing inside the execution itself.