> Many security tools for doing security audits with LLMs are based around a "look at this file" loop
This makes sense. At most it converts the question of ability to a question of cost (i.e. fire up a prompt for each file).
This article reduces the hype about Mythos in my mind: A new model that can find 9 new bugs while no previous model can identify them, is a whole different story from what this article demonstrates: that only 2/9 of the detected bugs are new for Mythos.
> You can see which bugs each model found in the full report.
This should have probably been surfaced at the text. To answer my own question: If one uses gemma4-26b-a4b, mimo-v2.5-pro and gemini-3.5-flash then 7/9 bugs are covered, while no model can uncover the remaining two.
> And, you have misunderstood what the benchmark does. It tells the model to audit the file, and it is allowed to look at the rest of the repo. It is not pointed at the bug.
I obviously meant that the model was pointed to the bug report. What would have been the point of providing the actual bug. This is still easier than telling Mythos to generally look at a codebase and find any bug.
Id soft folks were using 2 graphics cards, an EGA/VGA and an MDA one to do multihead debugging. It was possible because the two card technologies actually mapped their frame buffers onto two separate address ranges. Cool stuff.
The "best" model finds 4/9 bugs. It would be interesting to see if all models find the _same_ bugs. Does a collection of models exist that can cover all 9?
Also, it seems to me that pointing a model to a bug and asking it to solve it is somewhat easier than what Mythos did, which if I understand correctly, was to generally look at a codebase and find any bug. Even so, non-Mythos models only managed to fix 4/9 of these bugs.
I think the article makes the point that Mythos is at a different level.
The Q10 has been my daily driver for years, great keyboard. I see that they're coming out with a split alice [1] though sadly it seems it is only for the Chinese market
I see your point, though strictly speaking, the two products I mentioned mimic 12AX7 tubes, which are preamp tubes. I'm not aware of E2E designs that also mimic power tubes.
An experienced guitarist cannot distinguish between "captured" amps, or amps which at their core simulate vacuum tubes at the software level. I definitely can't tell the two apart. However, I believe it is easy to distinguish a pure vacuum tube-based circuit from a JFET/MOSFET-based one.
There do exist vacuum tube replacements like the AMT 12AX7WS [1] or Jet City's RetroVales [2], but I would argue that the fact that they try to emulate tubes via transistors is a strong indicator that the natural circuits for both sound distinct enough for guitarists.
Sup-par phrasing is a subtle advantage of non-AI generated text. In the past, I would be put off by this bad phrasing and the typo ("requier") in the text, but these days, it's a signal that a human took the time to write this, which makes me happy to see.
After all these years, I still think that this was a wasted opportunity on Google's side. As is well-known, Google (Googol) is a constant: 10^100. Instead of Alphabet, they should have named the umbrella company AlephBet [1] with a tag line: "we stopped thinking in constants".