I’m skeptical that the problem they are trying to solve is truly unreasonable bandwidth demands.
Sometimes it feels like what people want is to only serve websites and content to good normal users but not evil bad “scrapers” (because maybe maybe your content will be monetized in some nebulous way) but … you put your content up publicly on the web! That should be part of reasonable use!
EDIT: Lwn.net is perhaps not a fair target of my ire.
“There is also a desire to not impede the operation of legitimate search engines, the Internet Archive, and other such groups. Some sites may add explicit allowlists to, for example, give the dominant search engine access to the site. Such measures have the effect of further entrenching a monopoly that already serves us poorly and should be avoided. We have, thus far, succeeded in that.”
I'm exploring the pi based options now and I like that Oh-my-pi actually adds new value over extensions (the advisor and stream interruption hooks are pretty clever).
I just wish it had a way for me to downlimit tool access (I love midsized local models, and I'd love to enable only 30% of the power for some usecases).
This seems like a responsibly designed service, but I find it a little odd and baffling that we need such intermediaries for the average hobbyist / small project to reliably access sets of content published on the internet.
In a real job, you would be allowed to see the test case that failed and tweak your code (or more likely the poorly written test).
If you let a modern LLM do even the first, they’d crush this specific benchmark.
What is interesting is understanding how LLMs are able to beat 70+% on this benchmark or getting some of the poorly framed questions right? Are they implicitly learning the test writers style? Are the solutions leaking into their training set?
Perhaps reassuring is that even Fable stalls out at ~72% (on the hidden set which OpenAI did not run this analysis on), so perhaps training on the bench is not happening in anything but the most indirect ways.
I care a lot because small open models can never learn idiosyncrasies like this, so I really want good ways to judge models fairly.
EDIT: Humm OpenAI is muddying the water a bit. Only 20%ish of problems are broken in ways that are unfair to the agent, 4-10% are broken in favorable ways, so the benchmark ceiling is probably closer to 80-85%
Could you please open source this or a 4B version? I’ve been messing around w hooking up vllms to cheap robots and skipping the whole ROS stack and this would be an absolute delight to play with
“Spritely has been going strong for many years. I suppose Hoot was created to make Goblins [0] available to a broader public via wasm, as a reference implementation of CapTP specification”
I threw something like this together w a simple browser front end, mostly because I like running mid to large open models but can’t trust them to not go insane. Will share at some point soon
It’s tragic that memory is so expensive, and yet, we don’t have chipsets to exploit the bandwidth possibilities of the memory being burnt on these devices.
Absolutely no reason that these need to be capped at 256 GB/s other than shortsighted design from three years ago.
Is there a good way to teach kids relative pitch (beyond exposing them to a broad range of music etc.)? I struggle with this and have tried multiple times to learn different instruments from different musical traditions and instructors and have mostly failed over the years.
I think the way this works is with an internal structure, that houses the plastic and is expected to deform, printed first (so it cools), then outer walls with perhaps some air gaping for insulation, then injection into the inner structure at the lowest temp possible, then the next level starts.
Would print slow but might be genuinely strong vs normal infill + many walls (weight for weight).
Multi head printers like the U1 or H2D could do even better with high heat deflection temp plastics like carbon ASA or nylon for the inner structure and outer walls and strong low temp PLA for the injection.