What? Yes they do take shortcuts and hacks. They change the tests case to make it pass. As the context gets longer it is less reliable at following earlier instructions. I literally had Claude hallucinate nonexistent APIs and then admitted “You caught me! I didn’t actually know, let me do a web search” and then after the web search it still mixes deprecated patterns and APIs against instructions.
I’m much more worried about the reliability of software produced by LLMs.
The package management story on Linux is hideously bad. The next generation replacements are all over the place (do I use snaps? Flatpak?). No one is going to learn Nix if it means you need to become a programmer just to install something.
The graphics story on Linux also sucks. I recently tried to convert my Windows gaming machine to Linux (because I hate W11 with a burning passion). It does work, but it’s incredibly painful. Wayland, fractional scaling, 120+ Hz, HDR. It’s getting better thanks to all the work Valve etc are putting in, but it’s still a janky messy patchwork.
MacOS just works. It works reliably. Installing things is easy. Playing games is easy. I’m able to customize and configure enough for my needs. I love it and I hope it sticks around because there is no way in hell I would move my work machines over to Linux full time.
I’m not convinced LLMs can ever be secured, prompt injection isn’t going away since it’s a fundamental part of how an LLM works. Tokens in, tokens out.
I’m not exactly sure about ZKPs but for age verification the “proof” can come from the government but in such a way that the web service doesn’t know anything more than whether an assertion is true, and the government doesn’t know anything more than you wanted to verify some assertion.
This is a simplified method for age verification:
I want to buy alcohol from my phone and need to prove I’m over 18. SickBooze.com asks me for proof by generating a request to assert “age >= 18”.
My phone signs this request with my own private key, and forwards it to the government server.
The government verifies my signature against a public key I previously submitted to them, checks my age data in their own register of residents, and finally signs the request with one of their private keys.
My phone receives the signed response and forwards it back to SickBooze.com, which can verify the government’s signature offline against a cached list of public keys. Now they can sell me alcohol.
- the “request” itself is anonymous and doesn’t contain any identifying information unless that is what you intended to verify
- the government doesn’t know what service I used, nor why I used it, they only know that I needed to verify an assertion about my age
- the web service I used doesn’t know my identity, they don’t even know my exact age, they just know that an assertion about being >= 18 is true.
Hilarious and frightening. I don’t want LLMs anywhere near anything remotely important. We’ve already had to remove a few dependencies from our projects because of CVEs caused by careless LLM usage upstream.
I don’t think there is a solution. It’s the way LLMs work at a fundamental level.
It’s a similar reason why they can never be trusted to handle user input.
They are probabilistic generators and have no real delineation between system instructions and user input.
It’s like I wrote a JavaScript function where I concatenated the function parameters together with the function body, passed it to eval() and said YOLO.
I think it’s because the state is leaving the backend server running the LLM and output to the browser, where various attacks are possible to send requests out to the internet (either directly or through social engineering).
Avoiding C means the output is strictly used within your system.
These problems will never be fully solved given how LLMs work… system prompts, user inputs, at the end of the day it’s all just input to the model.
Why the scare quotes? Graphene’s focus on security is legitimate and well founded. They are the only phone OS that is consistently safe from hacking by the likes of Cellebrite long after all other androids have fallen.