That's what SlopStop is for :) developing methodologies that scale for detecting slop.
I mean it sounds like that already has a lot of overlap with our Small Web indexing efforts, so that part of our indexing efforts could be an extension of that. A lot if this is still in development though so I can't speak on specifics just yet.
> didn't realize Kagi had no aspirations to build their own general purpose index
Kagi employee here. We're actively working on building our own indexes beyond the limited ones we have now, not just a general index but also purpose built indexes for things like programming, etc.
There's a complete open source toolchain available for ice40 and ecp5 FPGAs from Lattice, for both Verilog and VHDL. I've been using it for almost a year now and it's really good. See github.com/YosysHQ