That's a great question - right now it is only looking at the results from a battery of several dozen indicators that we compute upstream of the model itself (which saves massively on tokens)
As small models continue to improve, and edge hardware becomes more capable, we would really like to run larger models that could incorporate full page content and screengrab data, which would be more likely to catch these kinds of attacks.
But we also find that sites that do one shady thing usually do others, which is a big reason why a tiny model like this can work - and why we are betting on low latency being a differentiating factor in real-world impacts.
CLAVIER is amazing, the wire system alone is such a huge improvement over ORCA, and it's now feasible to make much larger patches and refactor safely, kudos to River for all the hard work on the polish and quality-of-life.
Love 100r! There aren't a ton of examples online, but their livecoding music software/language, ORCA, is a remarkable instrument.
https://100r.co/site/orca.html
As small models continue to improve, and edge hardware becomes more capable, we would really like to run larger models that could incorporate full page content and screengrab data, which would be more likely to catch these kinds of attacks.
But we also find that sites that do one shady thing usually do others, which is a big reason why a tiny model like this can work - and why we are betting on low latency being a differentiating factor in real-world impacts.