It's not quite as bad as the parent made it out to be, the largest I've seen is 32kB per token (where sometimes, a token represents a byte, but usually it represents more than one.)
It's forced by the nature of how LLMs use vector embeddings for language.
Basically, a single token in a LLM is represented as a n-element vector, where n is the "hidden dimension", also known as model dimension. In order for the model to be smart, the hidden dimension needs to be large, on the order of 2^16 on top-tier models. Elements of this vector are typically quantized to 2-byte floats, or sometimes smaller. Every possible fact is embedded as a direction in this very many dimensional vector space, and a token is related to a fact if the vector representing that token points into a similar direction as that fact. You can do vector math about these things, famously for most trained models, if you find the vector embedding for king, man, woman and queen, and calculate king - man + woman, the result is very close to queen.
(Does that mean that there are 2^16 possible different kinds facts about things in this model? No, because high-dimensional geometry is very unintuitively powerful. The facts are not axis-aligned, and they don't need to be perfectly non-orthogonal. This matters, because the numbers of individual vectors you can fit into a single 2^16 dimensional space that are orthogonal with each other (all angles 90degrees) is of course 2^16. But, if you allow for almost orthogonal vectors, the number is larger than the amount of atoms in the universe. If this sounds wacky, for people with a CS background it can help to think it working a bit like a bloom filter, in that collisions are possible. Although in actuality they are theoretical, because 2^16 is a very large number.)
That's not how modern ethernet works at all. A single NIC talking directly to an another one has no collisions ever. Depending on what your channel is, either you have separate wires for the directions, or you are using a hybrid circuit (as in telegraphs, the term is so overloaded it's hard to google). Either way, packets going in one direction never wait for packets going in the other.
This is an example of common knowledge that is wrong. People look at their cash burn, assume that they spend this to subsidize inference, and get bonkers answers. Inference is not their largest expense.
Inference is cheap. Anthropic is only drastically subsidizing their plans if you count their training expenses as part of their costs.
Mostly training. Claude didn't just get to be so good at coding by magic, it was suddenly so good because they did truly staggering amounts of RLHF and RLAIF on it. They are still doing that today, on any tasks they can figure out how to evaluate it on. This is capex for them.
Their margins on inference are >90% today for tokens they sell (plans are hard to count, but still profitable). Based on what we know of it's size and architecture, running Opus is not more than 2x more expensive than running Deepseek v4 pro, for which tokens are available at under 10% of the cost of Opus. Again, the reason their margins are 50% is because they are spending so much on things that are not inference, not because inference is expensive.
> The cheap model providers have a much better chance of achieving that.
Anthropic can do it with a push of a button, once they calculate that it will provide them better profit than current pricing.
> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo?
They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices.
> Are these models cheap because they are open weight and having hundreds or people stress test running them on different hardware helped to lower the cost? Or is it that they are being provided as loss leaders to drive the prices down?
Neither. They are cheap because they have neither technical edge nor brand power to keep the prices high, and so have to ask commodity prices for them.
People somehow still don't get it, despite everyone who studies the economics of it telling them: Inference is dirt cheap. Training is expensive, inference is cheap, and getting cheaper.
It won't make sense to run them after two years. The vendors will be limited on datacenter space, power and cooling, and there will be new hardware available that will run the same models at a fraction of the power.
A100 -> H100 was >3x tokens per joule, H100 -> B200 >10x. There are significant low-hanging fruit still available in architectural efficiency, and the vendors are chasing them.
This is the big risk for AI companies that I feel is not being sufficiently priced in. Almost none of the investments they are making are durable, the depreciation schedules for everything but the real estate should be less than 24 months. Until the hardware is stable enough that you only get double-digit % improvements per generation, it should almost be counted as opex.
All of them. And it's not just an os thing, it's usually printed in the bottom right corner of the M key, the same way € is in the corner of the E key.
Semitic, not Hebrew. Hebrew is one language in the semitic group, alongside Arabic, Amharic and many more. They were much more spread out in the west before the iron age, with most people in Asia Minor belonging to the group. Some of the earliest states used the languages, and they spread alongside the idea of states.
The sentence after it explains. The core of "Reinventing the renaissance" is basically the renaissance-related blog entries from https://www.exurbe.com/ , developed further into a book.
> Most models assume that a "Dunkelflaute" (span of time with significantly reduced solar and wind output) will last at most 10 days.
The longest recorded in Finland is 90 days. More than two weeks of it continuously happens nearly every winter.
> as the entire European electricity net is synchronized
It is not. The CESA is synchronized. The various peripheral areas are not part of it.
> Power transmission is a thing.
It is not a thing you can trust. We have only just gotten a very sharp reminder of that. We have a neighbor that likes to cut sea cables as a fun past-time activity.
> you can convert electricity into h2 or methane
I am very pro that, but this will take a very long time to build out.
> solar and wind anticorrelate more than you think
They anticorrelate in some locations. In others, they don't. Here in Finland in the winter you get effectively zero sun. We also get persistent stationary anticyclones. That means potentially over a month of temps in the -30°C region, and zero wind.
Australia is extremely sunny. California is even better, they are modeling that assuming they keep their current hydro capacity, they only need to add ~3h in batteries. Hot places also do better than cold places, because the usage peaks track the sun.
> In Europe or America you might need 7-8 while in carbon industry PR models (the same people who denied global warming) seem to think you need 300+.
How on earth do you expect 7-8 to be enough? 300 isn't enough either. The real number for a fully renewable-based grid here is somewhere north of 2000.
Renewables are great in some situations. There are places in the world that should go for 100% renewables as quickly as possible. It also makes sense to locate a lot of the high-consuming industry in such places. But before you hawk your solution everywhere, you need to actually study the local conditions, and not try to extrapolate anything from Australia.
Lore itself is not an example of a program that meaningfully benefits from any of the key features of Lore.
Lore is meant for situations where your repository is going to contain gigabytes of binary files, such as art assets for games. Git is technically great at everything but that, and even the external solutions for that situation still kind of suck.
For a lot of people, because they already have copper in the walls.
You are correct that 10GBASE-T really shouldn't be the default choice, fiber and DAC both have advantages over it. But compatibility is important, and there are a lot of situations where 10GBASE-T is just more convenient.
The format for all modern video codecs is not the kind of format where any specific piece of uncompressed input should always be encoded the same way, but more like a very restricted programming language that gives the encoder a lot of tools to compress the video, and which tools they use and how they use them are up to them.
It's forced by the nature of how LLMs use vector embeddings for language.
Basically, a single token in a LLM is represented as a n-element vector, where n is the "hidden dimension", also known as model dimension. In order for the model to be smart, the hidden dimension needs to be large, on the order of 2^16 on top-tier models. Elements of this vector are typically quantized to 2-byte floats, or sometimes smaller. Every possible fact is embedded as a direction in this very many dimensional vector space, and a token is related to a fact if the vector representing that token points into a similar direction as that fact. You can do vector math about these things, famously for most trained models, if you find the vector embedding for king, man, woman and queen, and calculate king - man + woman, the result is very close to queen.
(Does that mean that there are 2^16 possible different kinds facts about things in this model? No, because high-dimensional geometry is very unintuitively powerful. The facts are not axis-aligned, and they don't need to be perfectly non-orthogonal. This matters, because the numbers of individual vectors you can fit into a single 2^16 dimensional space that are orthogonal with each other (all angles 90degrees) is of course 2^16. But, if you allow for almost orthogonal vectors, the number is larger than the amount of atoms in the universe. If this sounds wacky, for people with a CS background it can help to think it working a bit like a bloom filter, in that collisions are possible. Although in actuality they are theoretical, because 2^16 is a very large number.)