Grok since it's likely to include the training data from over a 100 years of autonomous driving + all the space tech included meaning that it might even have some rocket-y stuff
Sympathy to engineers and everyone at github, it's good that they're being open even if findings are limited. I'm sure they will figure out the root cause and will publish results to be a learning experience for everyone else
I have a Mac Studio with 512GB Ram and ran models of different sizes to test out how local agents are and I agree that local models aren't there yet but that depends on whether you need a lot of knowledge or not to answer your question, and I think it should be possible to either distill or train a smaller model that works on a subset of knowledge tailored toward local execution. My main interest is in reducing the latency and it feels that the local agents that work at high speeds should be an answer to this but it's not something that someone is trying to solve yet. Feels like if I could get a smaller model that could run at incredible speed locally that could unlock some interesting autoresearching.
Since the classifier would need to have access to the whole log message I was looking into how search is organized for the CLP compression and see that:
> First, recall that CLP-compressed logs are searchable–a user query will first be directed to dictionary searches, and only matching log messages will be decompressed.
so then yeah it can be combined with a classifier as they get decompressed to get a filtered view at only log lines that should be interesting.
The toughest part is still figuring out what does "interesting" actually mean in this context and without domain knowledge of the logs it would be difficult to capture everything. But I think it's still better than going through all the logs post searching.
yeah, I saw Claude Code doing lots of grepping/find and was curious if that approach might miss something in the log lines or if loading small portion of interesting log lines into the context could help. I find frequently that just looking at ERROR/WARN lines is not enough since some might not actually be errors and some other skipped log lines might have something to look into.
And I just wanted to try MCP tooling tbh hehe Took me 2 days to create this to be honest
the patter matcher is interesting to also collapse log lines and compare that between runs, thank you!
In my tool I was going more of a premise that it's frequently difficult to even say what you're looking for so I wanted to have some step after reading logs to say what should be actually analyzed further which naturally requires to have some model
Lots of logs contain non-interesting information so it easily pollutes the context. Instead, my approach has a TF-IDF classifier + a BERT model on GPU for classifying log lines further to reduce the number of logs that should be then fed to a LLM model. The total size of the models is 50MB and the classifier is written in Rust so it allows achieve >1M lines/sec for classifying. And it finds interesting cases that can be missed by simple grepping
Interacting with this model is just supplying your data over to an adversary with unknown intents. Using an open source model is subjecting your thought process to be programmed with carefully curated data and a systems prompt of unknown direction and intent.
It could just be a side effect of the more focused nature of the front end web sites - the more you focus on the content, the more you feel the less you feel the need for it.