If it is to be believed—which I wouldn't count on—it sounds like the intention of the prompt was to keep the model focused, but the model's interpretation of it was to keep you focused.
With a car, when I turn the steering wheel a quarter turn, the car immediately starts to turn at a known rate (I can't quantify it, but my muscle memory knows it). With an LLM, if I ask it to "turn the car", my "steering wheel" is natural language. I have to spend effort to figure out how to convey the rate at which I want the car to turn, and because the LLM has a noticeable delay, I also need to preemptively include some language anticipating how the expected rate should feel and what to do if it doesn't seem to feel right. And then the next time I say the same thing, the LLM decides to steer the car a bit differently anyways.
This matches my experience coding, and the expectations I (and many other) have been voicing for years about why AI is still a ways away from solving coding. The hard part of programming isn't pressing the keys to put code into a file, it's understanding the problem and coming up with a good, clean, extensible solution. AI is great at making ok solutions that technically work, but pretty bad at having a good vision for a solution. AI is still very useful for bouncing ideas off of and exploring to create a better understanding, but writing a good spec for an LLM to implement isn't much easier than just writing the code yourself.
I'm also curious, specifically about the cost of training vs inference, and comparing that to other industries that can have high R&D costs. My instinct says that open weights aren't feasible because of the obvious issue where there is no incentive to develop your own model rather than just taking someone else's model. However, I could see a scenario where a hardware company designs a model that is open weights but optimized strongly for their own proprietary hardware, cutting their costs of inference low enough to be competitive with a hypothetical other company that doesn't have any R&D expediture.
It's a bad aphorism if you take it literally and don't think about any nuance, but it's the correct priority. In many cases each of the three steps can take longer than the last, and each step is useless if the previous step isn't possible. The lesson is supposed to be to focus on each step at a time, not to completely ignore the other steps.
If you're making a complicated webapp, use your favorite framework to make it functional, and then if it's functional and not already fast enough, look at the slowest parts and replace them with faster alternatives. It's not going to result in the most elegant solution, but in most cases it will be good enough. Better to have something that works than to spend an extra year reinventing the wheel.
I have a similar anxiety about emailing or messaging people directly. If I'm just posting a comment on HN or reddit, there's not much commitment and if someone responds but I don't feel like continuing the thread, it's fine to completely ignore it. But if I send something directly to someone, it feels very personal, and like I'm starting a conversation. Once I'm in a conversation I'm obligated to continue it until it reaches a conclusion, and while the anxiety of trying to write things in a personal way is bad, the anxiety of trying to force a conclusion to the conversation is worse.
This feels like less of a gamma issue and more of a color space and monitor calibration issue. Possibly affected by the fact that oklab didn't exist when this article was written and the general awareness of color spaces was lower.
As someone with no plans to live or vacation in the caribbean, I'm curious. Is there a specific notable reason, or is it just a combination of littler things (cost, convenience, politics, weather, etc.)?
Part of the premise of the article is blatantly wrong. Distillation prevention was always visible. The only invisible safeguard was against frontier model development like development of training pipelines. This doesn't change the general idea that invisible degradation is bad and has been reverted, but the article changes the framing of the original issue from "preventing accelerating AI in the future" to "preventing cheaper AI right now".
Our logic at the time was that the relatively fixed cost of figuring out the hardware and developing device-specific software was less than the variable cost-per-board delta of like $20.
Huh. I had a work project a decade ago where we were evaluating SBCs as drivers for kiosks. At that time, the prevailing wisdom was that the Pi was specifically not for industry, as its main advantage was the strong community to provide support for DIYers. Competitors like PINE64 and Orange Pi were the same/better specs at half the price.
This misses the point of the discussion. Yes, we can understand what it is like to be a human with echolocation. However, we can't understand what it is like to be a bat.
> Now, we all know that it can take a while to find a long sequence of digits in π, so for practical reasons, we should break the files up into smaller chunks that can be more readily found.
> In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in π.
I work on "AI" stuff. Not LLMs, but large neural nets that include transformers and are as big as the smaller LLMs of today. Half the prompts I give fit their category of examples like "building pretraining pipelines, distributed training infrastructure, or ML accelerator design." I generally don't trust AI and have been very slow to trust and adopt it, but recently I've been warming up to it as part of my coding workflow.
Now with this, it makes me wonder if I should step back? Should I try to get used to a non-claude model/harness? Should I go back to less AI in my workflow? Either way, it makes me less inclined to pay for tokens from claude.
In addition to what mbauman said, hair follicles and the hair itself are not single-cell. I can't immediately find the composition and average cell size, but even a long and thick strand of hair is less than 2 orders of magnitude larger than the largest neurons. I doubt any individual hair cell is very large.