This whole blog post makes me sad. I've been active on both sides of the vulnerability disclosure process for well over a decade and have reported a whole bunch [1] of security bugs to the Go security team. I was there back when Filippo was running the show and have continued since Roland took over. My experience with the people there has always been great.
> Ultimately, it all stems from our responsibility to our users. The security researchers are not special, the insight and confidentiality are, and we need them to keep our users safe. Ignoring a security report communicates you don’t care about users’ security, and it’s rightly a reason for shame.
100%. This was always true and I still think it is. LLMs don't change anything. At most they shift the balance and force a temporary compromise.
> LLMs are as good as almost any security researcher
This statement is extremely dependent on the definition of a security researcher. It might hold if you consider anyone with a HackerOne account, but if you restrict the definition to people who actually put in some effort, it's just not true. LLMs can find some real vulnerabilities, yes, but they also spew unprecedented volumes of garbage that an expert can immediately recognize as such.
> The insight is not scarce and precious anymore. The bottleneck now is not finding potential issues but assessing which ones are real.
Assessing which ones are real should be part of the insight. Real researchers will not submit 150 pages of spam, and three real bugs hidden in 150 pages of spam are not insight. In most cases a researcher will spend significant effort on triage before submitting anything, and an LLM still cannot do that reliably.
> Confidentiality, embargoes, and coordination also don’t matter nearly as much as they used to.
I'd argue these now matter more: the one thing LLMs do seem to do fairly well is figure out specific things based on sufficient information and a scope that's limited enough. So a plain commit containing a security fix is now much easier and cheaper to turn into an exploit than it was before.
> The years of vulnerability reports being special might be over, as weird and uncomfortable as that feels.
I'd hope not. Bug bounties might be over unless someone can figure out the spam problem, but disclosure programs that don't offer monetary incentives are probably just going through a tough period that will eventually calm down as the operators of LLMs realize the costs and do the math.
Unreliable reports have always been an issue and will remain one, LLMs or no. When it gets worse, like in the current influx of LLM-generated reports, the focus should be on identifying reliable researchers, building relationships, and providing guidance on how they can make the reports easier to triage.
Researchers are not special, but the insight they can provide totally is. LLMs might force everyone to make better use of that insight, instead of just consuming bug reports and drowning in triage.
The weAudit VSCode extension [1] works pretty well. It's designed for security work, but there's no reason why you couldn't use it for general note-keeping.
The big problem with transparency logs is that they can't prevent attacks in real time because of the merge delay. You'll only find out afterwards if you've been attacked. It significantly raises the bar for an attack, but can't stop one from happening.
Thanks! Automatic updates are still possible; you can implement a code signing-based flow on top of this, or fetch hashes from GitHub releases, or anything, really. Attacks are only possible during setup, and targeting at that point in time is difficult because the client won't have authenticated yet. Anything else (attacks that rely on clearing the local state) can be mitigated using careful UI design.
I've looked at web bundles and a variety of other solutions myself, but the service worker approach feels like a winner so far. There's no magic, nor any bug being abused, but the client does have to trust the server to behave nicely during initial setup. After the initial setup is done, the client never again has to trust the server again as long as the browser's local storage isn't purged manually; so if the server is compromised after the initial setup, the compromised server cannot compromise established clients. It's not perfect, there's still the need for initial point-in-time trust, but it's still a significant improvement on the standard way of serving webapps where a server can compromise any client at any time.
The way it works is the server returns a unique service worker script every time, and the script file itself contains an AES key. The user trusts the server not to store this key and the server never sees it again. This AES key is then used to encrypt all persisted local state and sign all cached source files. If the server replaces the service worker, the key is lost and local state cannot be accessed. If the server somehow replaces a source file, its integrity check will fail and the webapp will refuse to load it. If the server manages to skip the service worker and serve a malicious file directly (e.g. because the user did Shift+F5), the malicious file won't have access to any local state because the service worker will refuse to give it access. The server can destroy all local state and then serve a malicious application, but the user will immediately notice, hopefully before interacting with the app, because suddenly all their data is gone.
This is great. I feel like E2EE has slowly fallen out of focus in recent years as the tech has stabilized, but important developments like this and the MLS standardization still continue to happen.
One specific area where I'd love to see more focus and attention is the web as a platform for E2EE applications. Currently, because of the inherent problems related to application delivery and trust relationships on the web, every step forward in E2EE adoption is a step away from webapps being first-class citizens -- even as PWAs keep becoming more viable for a wider range of use-cases otherwise. Even though an increasing number of companies maintain web implementations of their E2EE apps, these are always the fallback option when nothing "better" is available; the tech to make E2EE secure in webapps doesn't exist yet, but companies also have a unrelated incentives to push users to native apps. There are no serious efforts to remedy the situation and develop tech that would make it possible to deliver secure E2EE through the web.
The post mentions a couple of relevant goals:
> 3. Control over endpoints
> 8. Third-party scrutiny
They also mention the Code Verify extension[1], which may seem like a solution, but does not stand up to scrutiny: It only notifies the user of unexpected changes in the app, but does not prevent them. The detection logic it implements also seems trivially bypassable, and in more ways than one. Even if it was sufficiently enforcing application integrity, an extension like Code Verify is unlikely to ever become widely-adopted enough to make a dent. And of course it's not even available in all browsers on all host platforms.
There are also other similar extensions that suffer from similar shortcomings.
Browser vendors could solve the problem by providing APIs that allow the kind of integrity enforcement needed, akin to SRI[2], but that would mean you first have to agree on a standard and then implement it consistently everywhere and then webapps could slowly start adopting it. And because of past failures like HPKP[3], browser vendors would probably be hesitant to even start considering anything like it.
I believe a solution is possible using only the currently available web APIs, however, and for the past few months I've been prototyping something that's now at a stage where I can call it functional. The general idea is that using service worker APIs and a little bit of cryptography, a server and a client application can mutually agree to harden the application instance in a way that the server can no longer push new updates to it. After that, the client application can be inspected manually with no risk of it changing unannounced, and new versions of the app can be delivered in a controlled way. While my prototype is nowhere near production-grade at this point, it's nearing a stage where I'll be able to publish it for public scrutiny and fully validate the concept. Until then I'll be implementing tests and examples, documenting the API and threat model, and smoothing out the rough parts of the code.
If anyone's interested in collaborating on this or just hearing more details, feel free to reach out. I'd love some early feedback before going fully public.
I'm trying to solve the problem of "how can I trust an e2ee messaging app on the web". Basically, the issue is that while e2ee messaging apps (think WhatsApp, Signal) assume no trust in the server, the user still has to trust the client -- and on the web, the server controls the client. Desktop and mobile platforms solve the trust issue in multiple ways: code signing, app stores, reproducible builds, publicly available hashes etc. On the web, none of that's possible. That's why Signal doesn't have a web client. WhatsApp does, but using it defeats the point of e2ee.
My proposed solution is to use Service Workers to cache a web app in the browser and employ clever tricks to prevent the server from pushing updates to either the Service Worker or the caches. This way the user can then control any updates and verify new versions using means that are already familiar from other ecosystems: comparing hashes, trusting only signed code, etc.
The goal isn't to develop a new e2ee messaging app. Instead I'm prototyping something that resembles an auto-update framework like Squirrel [1], only for web apps. Ideally it will be largely "plug and play", i.e. you could take the existing WhatsApp web app, serve it using the updater framework, and your users would now have a trustworthy version of WhatsApp.
So far I have a small amount of PoC-level code validating several small parts of the larger concept. For instance, I'm fairly confident that I will be able to reliably prevent forceful server-controlled updates, which is a core requirement. Right now I'm in the process of formalizing a threat model, hoping to spot any gaps before I move forward with the implementation.
Feedback on the idea in general would be highly appreciated, but I'd also love to hear any more specific concerns regarding technical solutions, UX, etc.
There's an even more ubiquitous app that also usually has mic and camera permissions and suffers from a similar (but technically unrelated) local code injection issue: Chrome. The bug is described here [0] and was closed as WontFix because "if your machine is compromised, it's beyond the scope of anything Chrome can do about it".
Even if you don't use Chrome, you probably have at least a few Electron apps installed; they all suffer from the same issue.
The only logical conclusion is the macOS privacy model, TCC, is doomed. There's always an app that has non-default TCC permissions and is vulnerable to some type of local code injection, and at that point any malicious app can also access those TCC-protected features.
Noteworthy in this security release: 7 out of the 9 issues fixed are stack exhaustion bugs, meaning something in the stdlib is recursing too deeply and with a large enough input the runtime hits its 2 GB stack limit. Unlike it says on the announcement, though, the resulting crashes are not actual panics, but fatal errors that you can't recover from.
Most of these are pretty easy to hit, too: App taking in XML files larger than a couple of megabytes? Probably affected. Decompressing untrusted gzip files? Yeah pretty likely also affected. Doing static analysis or linting on Go source code? Definitely affected.
Blog author here; Russell's implementation is backed by github.com/beevik/etree, but like you said, it's just an interface. The tokenizer is still encoding/xml.
Adding better support for namespaces and providing APIs compatible with dsig doesn't remove the underlying vulnerabilities.
Just to be clear, I haven't really disclosed anything publicly, not regarding the e-payment API issue or any other issues for that matter. The SlideShare from my comment references the e-payment API vulnerability but doesn't disclose any technical details. It's not possible to reproduce the attack based on the slides alone.
The thing about setting deadlines like that (blind drop or not) is that it's very easy to look at it as some form of extortion. "This guy has cyberweapons, and unless we do what he tells us, he's going to release them on DATE. Better call the lawyers."
Also a big issue here, as with many software vulnerabilities, is that the people the public disclosure would actually damage are the users, not the company making the vulnerable software. The bank would only start losing money if the users (personal customers, business customers using their APIs) would notice the hack and start demanding their money back.
In Finland, most online stores allow you to pay for your shopping directly using your online bank. The way it works is the online store calls the bank's e-payment API, which in turn lets the user authenticate using their normal online bank credentials and accept the payment.
A few months back I did some research [1] on these e-payment APIs and noticed that one of the major banks had a serious flaw in their API implementation. It was possible for the end-user to manipulate the signed API calls to change the payment amount, effectively paying less than the actual price for products they buy.
I reported the issue to the bank and got a swift response where they acknowledged my report and said they were looking into it more closely. A few days later I got another email where they basically said "ok, this looks bad, and we can see it's pretty trivial to exploit, but... it's too expensive to fix, so we won't do anything".
I wasn't comfortable with this, so next I reported it to NCSC-FI/CERT-FI. They also agreed that it looked bad, but said that they had no way of forcing the bank to take action. So that got me nowhere either. I haven't heard from either NCSC-FI or the bank since, but the issue does appear to be partially mitigated now.
I've since found several other issues in the same bank's systems but haven't bothered to report them since they don't really seem to care.
The current Firefox extension platform is a mess. There are XUL add-ons, Jetpack add-ons, and SDK add-ons, nothing is fully compatible with E10s, and to say the security features are lacking would be an understatement.
The Chrome platform on the other hand was designed to be multi-process from the ground up, has some solid security research behind it (like actual academic studies), and generally just makes sense. The Chrome extension system (or something very similar to it) is also already supported in a bunch of other browsers, so why would Mozilla not use it in theirs?
Not any time soon. They're aiming for maximum compatibility with Chrome extensions, and Chrome doesn't have anything like that, so I wouldn't expect it to be a priority in Firefox either.
> Ultimately, it all stems from our responsibility to our users. The security researchers are not special, the insight and confidentiality are, and we need them to keep our users safe. Ignoring a security report communicates you don’t care about users’ security, and it’s rightly a reason for shame.
100%. This was always true and I still think it is. LLMs don't change anything. At most they shift the balance and force a temporary compromise.
> LLMs are as good as almost any security researcher
This statement is extremely dependent on the definition of a security researcher. It might hold if you consider anyone with a HackerOne account, but if you restrict the definition to people who actually put in some effort, it's just not true. LLMs can find some real vulnerabilities, yes, but they also spew unprecedented volumes of garbage that an expert can immediately recognize as such.
> The insight is not scarce and precious anymore. The bottleneck now is not finding potential issues but assessing which ones are real.
Assessing which ones are real should be part of the insight. Real researchers will not submit 150 pages of spam, and three real bugs hidden in 150 pages of spam are not insight. In most cases a researcher will spend significant effort on triage before submitting anything, and an LLM still cannot do that reliably.
> Confidentiality, embargoes, and coordination also don’t matter nearly as much as they used to.
I'd argue these now matter more: the one thing LLMs do seem to do fairly well is figure out specific things based on sufficient information and a scope that's limited enough. So a plain commit containing a security fix is now much easier and cheaper to turn into an exploit than it was before.
> The years of vulnerability reports being special might be over, as weird and uncomfortable as that feels.
I'd hope not. Bug bounties might be over unless someone can figure out the spam problem, but disclosure programs that don't offer monetary incentives are probably just going through a tough period that will eventually calm down as the operators of LLMs realize the costs and do the math.
Unreliable reports have always been an issue and will remain one, LLMs or no. When it gets worse, like in the current influx of LLM-generated reports, the focus should be on identifying reliable researchers, building relationships, and providing guidance on how they can make the reports easier to triage.
Researchers are not special, but the insight they can provide totally is. LLMs might force everyone to make better use of that insight, instead of just consuming bug reports and drowning in triage.
[1] https://groups.google.com/g/golang-announce/search?q=juho