Unless we do our own benchmarks, we have to take all the marketing fluff from the frontier labs at face value, and all public benchmarks degrade eventually as labs optimize towards them. OP’s approach is wasteful because it is brute force, but post says that an ELO is kept, so this is also an experiment, and I don‘t see what‘s wrong with that. You learn which model performs well in which settings which may save resources later. It‘s also wasteful to keep working with the wrong model/harness/tools for too long.
It sounds like it solves the problem that everybody who vibe codes over multiple projects runs into, but it does not provide evidence that it actually works and that it is better than structured collection of md files. I think what is lacking in this field are benchmarks like LoCoMo for long sessions.
Among the inexpensive models (and I include Grok 4.3 in this list), GLM 5.1 really sticks out!
On my personal test bench, when compared to other inexpensive models, GLM 5.1 provides the answers that I would consider most complete or satisfying (these are subjects that I consider myself an expert in). The answers tend to be more comprehensive, nuanced, and include references that I would consider the correct ones (if given access to web search).
I also find it a joy to code with, somewhere between Sonnet 4.6 and Opus 4.6 (have not tested Opus 4.7 yet).
I think if we want to build on what we have, instead of compaction at the end of the context window, the LLM would have to 'sleep', i.e. adjust its weights, then wake up with the last bits of the old context window in the new one, and have a 'feel' for what it did before through the change in weights. I just sense it's not that simple to get there, because simply updating the weights based on a single context sample risks degrading the weights of the whole network.
I like the idea of using small local model (or several) for tackling this problem, like low rank adaptation, but with current tech, I still have to piece this together or the small local models will forget old memories.
As long as there's no solution to the long-term memory problem, we will have a "country of geniuses in a data center" that are all suffering from anterograde amnesia (movie: Memento), which requires human hand-holding.
I have experimented with a lot of hacks, like hierarchies of indexed md files, semantic DBs, embeddings, dynamic context retrieval, but none of this is really a comprehensive solution to get something that feels as intelligent as what these systems are able to do within their context windows.
I am als a touch skeptical that adjusting weights to learn context will do the trick without a transformer-like innovation in reinforcement learning.
I guess you can make that same argument about USA and America. Canada is clearly America but a Canadian would not refer to himself as American whereas a US national would. Europeans hardly refer to themselves as such but when European countries are lumped together, it has become common to ignore geography and refer to those affiliated with EU membership or bilateral EU affiliation (Norway, Switzerland, Iceland) as European.
I really hope they continue to show some spine against this administration and do not allow to weaponize AI against human beings. It's the morally right thing to do!
Isn't it sad that we now have Russian, Chinese, American, European, etc alternatives? I mean I get it, Sept 11 paved the way for FISA orders and NSA overreach, Russia and China reverted back into dictatorship, but Europe is also at the edge. Shouldn't we rather fight that nationalistic power grab that just makes us all poorer and less free? And instead propagate global alternatives that are not subjected by some power-hungry state-/capital-sponsored overlord?
This is a perfect example of how to lie with statistics. All of these countries are either tax havens or oil-rich economies, apart from half of them having the population of a small city. The economic policy implemented by any of these countries cannot be implemented by a large economy with little or no natural resources, or would you recommend to Germany or Japan to just "HAVE" oil or open their banks as offshore foreign accounts?
Why Bedrock? Get the kids a Steam Deck, Prism Launcher, open a local server and boom :) It‘s not iOS convenience but they‘ll sure love to tinker with all the mods you can install.
Haha… no bot here. Been using Kagi for years now. Not sure what you‘re searching for. My own tests, admittedly early, found no instances, where Google gave better results.
Indeed what I do as well; gives me apps for Youtube, Netflix, etc. The only downside is that you have to login if you do not use the "app" for a while. Would Electron get around this?
Suppose you have a liberal mindset and work there, you must bend the knee and practice anticipatory obedience, or why else would you tell the world that the rocket will be ”dropping into the Gulf of America?“
It‘s still a stark abuse of power and borderline extortion by Google to use a private sentencing mechanism rather than dragging the purpetrator to public court over advertising and/or encouraging criminal activity, which may or may not have happened if Google Ads and Youtube were not part of the same monopolistic entity.
Did they use the online Deepseek Chat or the open source model. If you ask either about the Tianenmen Square you get very different answers, which may be true for response quality as well.
L1-regularized autoregressive features, holiday dummies, Fourier terms (if suitable in combination) yield lower test errors, are faster in training, and easier to cross-validate than Prophet.