The point in the article that you are not considering is how easy it is to switch to a different model provider right now.
You literally change a couple of env variables and you are done, your user experience is basically the same. I can try new models for an hour and be sure I can go back to the original model as quickly if I want.
That is not the case for the software you talked about. They all require way higher switching effort with more perceived risk.
Following the original comment concepts, if every model requires a different prompting technique to maximize its output, how can a benchmark based on sending the same prompt to all models be accurate?
We should create different prompts for each model, but then how reliable and unbiased can the benchmark be?
How often do you really need to look at that info while doing normal work?
Because to me and to the very vast majority of git users it is totally irrelevant.
It is nice that the info is available, but the more sane default would be to hide under a verbose flag not the other way around.
Imagine typing cd folder/ and have the whole filesystem subtree be displayed in the terminal. You are free to ignore it, but it is useless and inconvenient nonetheless
I think in this case the human effort was put into the actual discovery, honestly I don't mind if AI helped him write the blog post if the result is enjoyable and not sloppy
> most of the bugs were in the JS code, so for most things, hot reloading is enough for E2E validation and to run just the right tests. No need to run a full build and test suite (which takes 10+ minutes); the CI can do this.
Have you tried adding this instruction to your agents.MD? Avoiding situations were the agent start running a loop is the main use case of the file for me
I'm guessing the greatest reason behind each provider creating and agent harness is that (a) there is not a clear winner still and (b) it is harder to switch models with a competitor, as you also have to switch harness
Aside from LLM architecture, that already is a complex issue, an issue is that training data is unstructured text.
An LLM able to structurally separate context and instructions, should logically need separated data to train, and we don't have it.
Moreover, while an equally powerful LLM architecture solving this may exists, there are no guarantees at all that we are able to come up with it in a reasonable timeframe.
Without some signals moving in that direction, the most pragmatic and realistic way of looking at the problem is that it will not be solved in the near future
User identity attached is not a solution, it doesn't solve anything if you have to pull in external data that you can't control.
Like in the banking world, you can make everything super authenticated, but if you have an API that receives the latest wire transfer YOU received with the message attached, you don't control the message content and it can be an attack vector.
Being authenticated/authorized is not the solution, it is data that the user can access.
Aren't small local models worse efficiency-wise? It means that every person must have a powerful enough machine to power a small model, and we are very, very far away from that.
The best solution, from an efficiency point of view, is to use smaller models on datacenters, requiring much less of them.
I still don't see solutions on how a normal person can become a mantainer though.
If all relevant open source projects close up their contributions, you can't enter the project anymore from an external point of view.
Almost all open-source public figures started by being interested in a project and submitting PR to it, until eventually either joining the project as core mantainer or creating a separate open source project. The path is now closed, and I don't see a way in, outside of creating a popular open source yourself
In programming forums like Hacker News people are incredibly detached from the average experience with technology, sometimes it is buffling.
Most non technical people I know asked questions to Google even before the AI overview. Instead of looking for the answer in seo-bloated articles, they find it in the overview.
I think google should improve in detecting the kind of query when I need a link that I don't remember, and deactivate the overview on those. If I search for "ryanair booking" I clearly need the url for booking a Ryanair flight, AI overview is useless
Most of the time I'm looking for something very specific that there are plenty of articles about, but clicking on the articles results in popups, banners and an unhealthy amount of scrolling to get to the answer.
AI overview provides me the answer instantly.
Think about suff like "does china borders afghanistan".
In those cases you can be confident that the AI overview is right, and saved you time.
If it is a complex or niche question I tend not to trust the overview and go straight for legitimate-looking results
Well it depends on the url. Usually shareable url where "anyone with the link may access that file" contain a random element that makes it hard to guess if you don't have it (e.g. an UUID).
In other cases the content is at easily guessable path, and that is a whole different story
I have seen lately a number of new productivity suite.
How is the Microsoft Office compatibility managed by these tools? There is a popular SDK providing the compatibility? I can't imagine everyone reimplementing the full compatibility layer