> 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
Cloud opposes switch inertia. To setup a complex system in a different environment is a complex operation. Changing AI provider is switching an endpoint.
In Catania 10 gbit Internet costs 35 euros/month and is available everywhere in the city and even in a big slice of the small towns around Catania. And indeed public incentives played a big role. But what I believe it is more interesting is that 1 gigabit was common like 10 years ago or even more. Infrastructure is a bit too important to be left to what the market believes will be profitable.
Btw the paradox I have is that my local lan is 1 gbit...
Europeans do a lot of stupid things, but I believe in light of all the scandals we saw in recent times, you can't explain EU behavior and choices without accounting for corruption. EU division and different level among the different countries of wealth, integrity of political sphere, and different cultural biases make us the perfect target for bribes in order to control votes and choices. Not just promoted by external actors. The Chat Control is a great example: everybody understands how bad this is, the arguments are mostly a shield to avoid revealing the real agenda.
I love coffee, I take 2 to 3 every day (Italian espresso, so very short, little total caffeine), however for some reason taking the decaffeinated one in the evening interferes with my sleep in a similar way than the caffeinated one. Any hint?
DwarfStar work in progress numbers: I see 14 tokens/sec generation, that slopes to 10 t/s with longer 10k or more context size. Consider that the indexed attention requires evaluating 2048 selected rows, 2x DeepSeek and with less compression, so the performances with larger contexts here to south faster. Prefill can be 180 t/s on small contexts to 150 t/s and less with larger contexts. I used DeepSeek v4 PRO in this conditions, it is usable but it is far from the 35 t/s 400 t/s prefill you get with DeepSeek v4 Flash 2 bit on a MacBook m5 max. But likely my implementation is yet not optimized enough, so a bit more performance can be obtained. I'm using 4 bit quants. The model is also definitely less sparse than DeepSeek v4, so it activates a bigger percentage of parameters. If it works decently at 2-bit, that would be a win even for machines where 4-bit fits, since this would mean 2x memory (equivalent) bandwidth basically for the routed experts.
Local inference needs really hard a 1.2 / 1.5 T/s memory bandwidth system with 512GB and 2/3 times the GPU compute of Mac Studio M3 Ultra, at an affordable 10/15k price point. A variant with 1TB memory would also be welcomed at 20k price point.
So to avoid those energy-hungry LLM companies from scraping your website, you force each browser to compute a lot of hashes in a necessarily energy-hungry loop, creating, at the same time, all the kind of accessibility problems?
They didn't freaked since the order was to still allow 350 million people using it: there is, in such large population, everything, including single persons very against the country, the government and so forth. If they really freaked they would say "we need to investigate, you have to retire the model". That would be a more defensible POV at least.
Interestingly if we assume 16 concurrent users, prefill drops to 600 t/s and generation to 61 t/s, and this starts to be dangerously near to M5 Max 35 t/s generation and 400 t/s prefill you get with DwarfStar in your own laptop (that you use for many other things) that costs ~6500 usd/eur.
They try so hard to do a polished presentation that everything is kinda fake and unauthentic. I don't understand how this attitude survived so many years.
There is a different way to look at this: that is, actually the Transformer is a minimal complication of what the based model is: in theory the neural network could be just a huge FFN, which is anyway the part of the Transformer that does the heavy lifting. But this would be impossibile to train both numerically and computationally, so the Transformer encodes enough priors for it to work: the causal attention, and the math tricks like the residuals and so forth. But the bottom line of all this is that the Transformer works because of the incredible semantical power of simple/huge FFNs.
What I say has nothing to do with efficient market hypothesis. Here the question is simpler: in small companies where there are competitors, who does the wrong choices will be seriously hit since customers will star preferring less slop and more reliability, if AI is mis-used. And companies that instead of firing, hire the folks that are "ideas people" and can use AI efficiently, and now how to control the quality of the output, will deliver more and better. For bigger companies: AI is driving salaries at a more normal level (honestly we want a bit too high, in recent years, even for people with a very low level of knowledge, didn't we?) and to marginally reduce total spending and not deliver the timeline they have, and are used to observe for years, will be noticed. Also companies in the past had a dangerous tendency to over-hire. I don't think now they will invert the direction and over-fire. I have the feeling many managers will instead reason in terms: what is today the great programmer fit? The one with low level knowledge of each algorithm, or the one that has good ideas and understands product, quality, processes, other than programming? And they will try to mix AI and people in order to have an edge.
Actually workflow impact in the world of software can be observed in weeks/months at max. And token spending too, is a voice that they see at the high floors. Also, there was never a strong willing in IT companies to reduce cost of work force: it is done sometimes, but it is more common to see them over-hiring.
I don't think this makes any sense. Companies with managers that think AI capabilities are superior will be replaced if they are wrong as the companies will perform very poorly.
Indeed this will likely happen in the future, but not today. I was experimeting with SSD streaming in DwarfStar for DeepSeek v4 PRO inference in 128GB systems (and Flash inference iwth 32/64). GPT 5.5 ran the whole night, I checked what it had accomplished regardless of all the hints I provided in the specification document. After reasoning on the problem I gave him the design fixes and the tokens/sec were 4x after 10 minutes. And this is true for every domain where the human babysitting the AI know a few things in that domain. However this is a moving target, and at the current rate, soon or later, indeed AIs will do much better than us in many domains.
Note that this is not related to Black Forest Labs Flux, the image synthesis models builders, and is instead related to a PCB AI authoring product called Flux.ai.
A few years ago, the probability of such shit reaching the Hacker News home page was near zero, because regardless of the merits, here was not full of normies that could not understand when a behavior is unacceptable (I'm referring to the violence of the language of the issue). And now, here we are, surrounded by people that can't tell the most obvious things.
Website: http://invece.org
Blog: http://antirez.com
My sci-fi book (English translation): https://www.ibs.it/wohpe-ebook-inglese-salvatore-sanfilippo/e/9791280845337 or Amazon Kindle
Twitter / X: @antirez
Bluesky: https://bsky.app/profile/antirez.bsky.social