We're maybe only 2 years away from really useful, relatively affordable local LLM usage. You can buy a 5090 PC for $5-6k but 32GB of VRAM really limits model sizes to ~31B. And that won't change (even with NVidia's next generation) because NVidia uses VARM as an aggressive market segmentation technique.
No, the hope really is these other platforms with a shared memory architecture. The DGX Spark won't be it because of the aforementioned market segmentation. So that leaves two players: AMD and Apple.
The AMD platform is still too low memory bandwidth, currently <300GB/s. For comparison a 5090 (or 6000 Pro) is 1.8TB/s and the M3 Ultra Mac Studio is ~900GB/s. Oh and B100/B200 uses HBM3e memory at ~3.2TB/s. The M5 Max in some Macbook Pros tops out at ~600GB/s. So you need access to better RAM and better CPU architecture for all this.
My great white hope is Apple. They have the market power to get memory and build silicon that coul dhave enough FLOPS to compete with NVidia's platform. They've started talking about it and I've seen rumors they're targeting the M7 generation (2028) for a huge leap. I'll believe it when I see it however.
But the point is, I think we'll be running 31B models at 100+tok/s on enthusiast hardware in 2 years and we'll likely be able to locally run 100-400B models, possibly larger.
I'm not sure that anyone under the age of 45-50 can truly appreciate just how big of a deal Terminator 2 was and how big movie releases can be. Like, nothing in the MCU era or the Star Wars prequels and sequels comes remotely close. Yes, they gross a lot of money but in terms of cultural significance, I've seen nothing close.
At the time I lived in a city when the local movie theaters would typically run major releases on 1, maybe 2 screens. Session times were like 11am, 2pm, 5pm, 8pm 6 days a week and I think 1 less on Sundays. This was before the age of smaller theaters in the large multiplexes so a big movie theater might only have 4-8 screens.
3 weeks after T2 was released, it was still showing on screens in my local movie theater for 12-15 sessions a day, even on Sunday, from 8am til midnight. I actually waited a couple of weeks for the hype to die down and went on an 8am Sunday session knowing basically nothing (because that's how things worked then) and the movie theater was still full.
The CGI was a big part of it. It has some fan service to it. My movie theater cheered when Arnie came out of the bar wearing the leathers and hopped on the bike. But it's not overboard. It's actually a really great story, which is kinda unusual for a sequel. Like, James Cameron really has to be commended for that.
But there was another aspect too and that was Linda Hamilton. This was one of the first mainstream big-budget movies that changed the way women were portrayed in film. Lots of people had posters of her wearing the sunglasses, carrying weapons, etc. It was actually a really big deal.
The 90s really was a golden era for movies. Like I used to go 1-2 times a week and just watch whatever was on, basically. I don't think I've been to a movie theater since Avengers End Game and even in the 2010s it was a 2-3 times a year thing max.
But it is amazing how much they did with CGI in the early 1990s for T2.
A lot of people end up commenting on current and potential conflicts clearly knowing nothing about logistics and it shows. I think this whenever I see "China will invade Taiwan". But it also shows up when people don't realize the Iran war is unwinnable and was unwinnable from the start.
First, China. 100 miles of open ocean separate mainland China and Taiwan. To invade you need to put boots on the ground. That means you also need armor. And then you need everything that goes with that. Food, medical, resupply, ammunition, a place to repair your vehicles and so on. You can't airlift this, even with air supremacy. It requires ships, a lot of ships. And that means Taiwan, the US and allies could, if they wanted to, wreak havoc with supply lines.
I've seen one estimate that it would take over a million soldiers for China to occupy Taiwan, possibly two million. China simply does not have that amphibious capability so this is a silly conversation. American politicians talks this up to give more money to weapopns manufacturers for weapon systems that are too expensive and don't work and also to propagandize the population into "China = bad". But it's not a real threat.
On a clear day from Calais you can see the white cliffs of Dover. I believe the distance at its closest point is 17 miles. Yet despite having an Army that exceeded 12 million soldiers, Germany was unable to invade Britain. This is big of a problem water was then. It hasn't changed.
Second, Iran. Iran is surrounded by mountains on 3 sides and the Persian Gulf on the 4th. There is nowhere where a US invasion could muster like they did with Kuwait (ie from Saudi Arabia). We saw the futility of this in the Iran-Iraq war, which was a complete stalemate (with over a million dead) despite an almost decade of the US supplying Saddam Hussein.
So with mountains on 3 sides, that leaves an amphibious landing. Again, the problem remains of where would troops muster? It can't be anywhere beyond the Strait of Hormuz because the Navy can't defend against drone and missile attacks at close range like that, let alone get thousands of supply ships through safely. So that leaves Pakistan or Oman. Both are in range of drone attacks. The US has essentially abandoned all Gulf bases because of the complete inability to defend them. How would the US defend a million troops and all the supplies they need, including landing craft?
Logisitcally, the operation would be as complex and large as D-Day. The US doesn't have that military anymore. The Navy has a pair of amphibious landing craft that could put 5000 Marines somewhere. Great. They'd die. For enough infantry, you'd need a draft. You'd need to build so many ships. It would take years and nobody has that kind of time.
The Joint Chiefs know this. It's why no US president has started a war with Iran despite the Israeli PM asking every one of them to do so since at least Reagan. It's only this president who took the bait of Mossad telling him how the regime would fall within days after a decapitation strike.
The entire Iranian military project is to resist the US military. Bases buried in mountains, cheap ballistic missiles and drones that can be mass-produced, a mosaic defense to avoid potential decapitation and air defenses that are seemingly good enough that the US, even with F35s, still has to stand off.
I bet you won't find a single serious military planner who thought this operation was possible.
So if the Iran war is still going on in 5 years the global economy has collapsed. I'm not sure people realize just how quickly this situation is going to be dire. 2-3 months more of this and certain countries are going to have rolling blackouts. We will be having shortages of avgas and huge price spikes in avgas, gas and diesel. Diesel in particular is going to have a massive impact on inflation.
Oil futures don't currently reflect this because a lot of players have exited the market because they're sick of being fleeced by Donald Trump who has bet big and then announced yet another fake ceasefire, at least a dozen times. We actually have a bunch of short positions, such that we risk a Gamestop like short squeeze.
Ukraine is harder to figure out because it's really difficult to get good information. We have people on one side saying Russia is on the verge of collapse (and has been for 3 years) while it's also clear that Russia still has a manpower advantage and Ukraine's army is facing desertion and a lack of soldiers to draft. It's also unclear what the real impact of strikes on oil infrastructure deep in Russia are really changing the battle lines and overall position. If anything it might just exacerbate the energy crisis brought on by the Iran war.
One of the first gravitational wave detections by LIGO was I think the merger of two black holes or maybe a black hole and a neutron star. It was over a billion light years away I think but was so energetic that it conveted approximately 5 Solar masses into energy in about one second. That's ~10^48 Joules. In 1 second that is ~10^48 Watts.
For comparison, the Milky Way has an estimate of 5x10^36 Watts so we're talking about the energy output, very briefly, of roughly a trillion Milky Way galaxies.
The other that gets me is amgnetars. These are neutron stars with an insane magnetic field. The strongest detected exceeds 1 billion Tesla, making is 30 trillion times stronger than Earth's magnetic field. Get too close and it would flatten atoms and ultimately break molecular bonds and rip electrons out of your body. Google seems to think that happens at ~1000km, which is pretty close to get to a neutron star but still, that's a magnetic field.
These things are quite rare and quite unstable. If you think about it, they must have a lot of protons to generate a field so strong, which means that the gravity is overcoming the strong nuclear force but also the electric repulsion.
It's really the plastic straws that are the problem. That and avocado toast. And in 1-2 billion years when the Sun has heated up the Earth such that it's uninhabitable anyway (or 100 years at the rate we're going), we can at least feel comfort in all the shareholder value we've created along the way.
You're saying we need to retain manufacturing capability and expertise. I agree.
I'm saying we need to stop being bled dry by private weapons manufacturers on cost plus contracts who are rewarded, by definition, by making the system as expensive as possible.
If we were truly interested in having an effective military, we would bring production in-house and focus on standardized vehicles and weapons systems with standardized interchangeable parts on production lines that can be scaled up if necessary.
- Reapers fly at a "medium" altitude, which is up to about 50,000 feet;
- They can fly up to 300mph;
- Despite being relatively high up they are relatively slow so lots of country have the military capability to shoot them down and have done so. This includes Iran and the Houthis;
- A typical payload is 2x GBU-31 JDAM (1000lb each). The explosive payload is roughly half that but these are relatively cheap (<$50,000) because they're barely-guided gravity bombs;
- They will also have 4x AGM-114 Hellfire missiles, which are precision-guided, costs $100k+ each but only has an explosive payload of under 20lb. When people talk about drone assassinations that becccame super-popular in the Obama administration, this is largely what they're talking about;
- You can also skip the JDAMs and just have 8x Hellfires instead.
Now Shahed drones aren't as fast, don't fly as high, aren't as precise and you know they're there (a lot of Hellfire attacks are a surprise). But they're incredibly cheap and they overwhelm missile defences easily just by sheer volume. In fact, we're using $1-4M Patriot interceptors to shoot down $20k drones. Obviously that's not going to scale when Iran can produce thousands a month and the supply lag for Patriots is actually years long. Supplies of certain missile defense munitions are suspected to be critically low already and will take years to replenish.
So where I'm going with all this is that Reapers and Sheheds serve different purposes but however you look at it, a $50M Reaper with $1M+ in munitions is WAY less efficient at deliverying payloads that a swarm of cheap drones, particularly when your enemy has invested a lot in missile defense and your $50M Reaper is vulnerable to air-defense systems from even non-state actors (ie the Houthis).
Put another way, $20k for a 50-100kg payload is incredibly efficient and the US has essentially been forced to evacuate all their Gulf bases because they're completely unable to defend them. Billions in damage has been done to these bases too.
The Reaper just isn't fit for purpose anymore. Use it against a more militarized opponennt (eg Russia, China) and they'll shoot those things down like it was a carnival side show.
In his 1961 farewell address, President Eisenhower warned of the dangers of establishing the military-industrial complex [1]. We are seeing the fruits of that now that despite an annual budget over $1T the US has been militarily defeated by Iran (and Afghanistan). We build $13B aircraft carriers that don't work [2].
Defense contracting is nothing more than a wealth transfer from the government to the wealthy. This is what unfettered cost plus contracts looks like. We ridicule the Russian military for their insane levels of corruption (eg paying for tanks that never get built and the generals pocketing the money) but really the same thing has happened here. The things get built but they don't work and the entire industry is built around hiring former Pentagon people who specialize in procurement.
It doesn't have to be this way. Some of the US military's past equipment was legendary. The M1 rifle and M4/M16 family were cheap, reliable and effective. The Jeep was legendary for its reliability. The original M1 Abrams tank is widely considered the best tank the military ever built. If you listen to anyone in the military they'll tell you the vehicles are constantly broken down, hard to repair, expensive to maintain and outright dangerous.
Every dollar spent on the military is a dollar not spent on roads, schools, bridges, hospitals and trains, things that would actually benefit people. We're bankrupting ourselves to enrich the shareholders of Boeing, Lockheed-Martin, Northrop Grumman, RTX Corp and General Dynamics for what exactly?
And the proposed "defense" budget for 2027 is $1.5T, a roughly 50% increase.
This is also why I laugh whenever anyone pushes the idea that China is the Big Bad [tm], for two reasons. First, they don't have to be. We just want their to be a scary enemy to justify all this. Second, if they were, they would destroy us because it would ultimately come down to military industrial capability and we would lose. Orders of magnitude lose.
I would say it's funny but really it's just true. Thunderbolt cables and a lot of other high bandwidth cables have chips in either end, which some people don't realize. And yes, these chips have to handle IO in the gigabit range. So these microcontrollers are a bit apples to oranges (just like my WCH570 vs 486 comparison, to be clear) but even with all those caveats, yes it's basically true.
I don't even know how you learn to do something like this. I was curious about the CPU. I found the specs for the WCH570 [1]. Intel released the 486dx4/100 in1994 (for $650). It's not really the same thing but a 100MHz (kind of) 32 bit CPU (that does have an FPU) is about as close as you can find to this (which has no FPU but it does have USB2.0 and 2.4GHz wireless, which is well beyond what Intel CPUs could do at the time).
And this thing is 10-13 cents ~30 years later.
A better comparison would be the ARM CPUs you can get fairly cheaply today (eg the Broadcom BCM2712 in the RPi5) but they're way more capable than the CPUs of 30 years ago. The BCM2712 for example is a 64 bit quad core 2.4GHz CPU.
I guess I'm just amazed at how far hardware has come because I'm old enough to remember just how amazing the 486 was at the time.
Whenever I see something like this, I'm always curious how the libertarians rationalize their world view. Because this is what they want: no regulations where companies can do whatever they want. And they will.
We're witnessing the looting of America. Every level of government seems increasingly dedicated to transferring wealth from the taxpayer to the wealthy. But even that's not sufficient. Apparently the wealthy also need to poison the land and people too for an uptick in profits. Why should they care? Capital is mobile. They'll simply leave whenever society collapses.
One thought I hate reading this is: do you need upward mobility?
It's a serious question because in an ideal (IMHO) society, people can have full and satisfying lives with security and family without becoming a CEO. In the US, for example, there's an obsession with "getting ahead" but, by definition, only so many people can get ahead. And why do they want to? Because, at least in part, a basic job in insufficient to make ends meet in most cases now. This is a form of coercion.
This is orthogonal to the issue of German social inclusion and forms of xenophobia (eg in the housing applications you mention).
Personally I'd rather in a society where everyone's needs are met and it's not a race against a rising tide where only 20% of the population are above it.
It's fascinating to see the effects of anti-China propaganda. There are plenty of stories about China cracking down on corruptiojn yet people need to do contortions to make this anti-China somehow.
There's no evidence that Xi Jinping is like Putin (who has enriched himself to abe unknown but expectedly high dgree), no evidence the military generals have enriched themselves with corruption (again, like Russia) or really anything else. Instead there's evidence that the likes of Jack Ma, a billionaire, are brought to heel, China has cracked down on so-called yin-yang contracts, sentenced to death people who messed with the baby formula supply chain and so on.
People really should reflect on why they're so willing to seek confirmation bias and why they have that bias at all.
Here's a tip: if you take anything China says or does at face value you will be more correct than 95% of the China "experts" that have entrenched themselves in the media and Western policy circles.
I firmly believe that we could solve a whole bunch of these problems by making it illegal to advertise to and target minors in online advertising on social media platforms. Remove the incentive and you'll get a lot better behavior.
The beauty of this is you can do this without age verification. How? These companies already derive demographics from behavioral and contextual information but also, preventing targeting is as simple as not giving an option in audience targeting for minors. We already do this, for example, with using race in housing ads, which is illegal (and yes, this was violated).
You need to identify and limit things that become a proxy for age and companies need to be punished for this. But the fact that social media companies would be suppressing advertising to minors anyway will really devalue this kind of workaround.
I think OpenAI, Anthropic and SpaceX are going to envy the dinosaurs because there's not asteroid coming for them, there's three:
1. There will be no moat around frontier AI models in the future. China is going to make sure that happens. It's a national security interest for them. DeepSeek was the first shot across the bow for that but it won't end with them. There are other labs and there are non-Chinese actors too. The stratospheric valuations depend on there being that moat; and
2. Nobody seems to be considering what the next generation of AI hardware is going to do with current hyperscalar investments. We're about to go through this with the B100/200 move to R100/200 but a lot of the investments are probably slated for that next-gen. But what about 3 years from now when the hypothetical X100/200 comes out and doubles FLOPS and halves performance-per-watt. What will that do to existing investments? Some people are delusional and think that they'll get 10 years out of GPUs when 10 year old GPUs (eg V100) are sold for scrap and 5 year old GPUs (A100) cannot run DeepSeek v4 Pro. And people think the A100 is going to get another 5 years of use? No; and
3. Local LLMs are coming for remote usage. You can buy a 5090 PC for less than $5000 currently but you're limited to 32GB of VRAM, which will comfortably run 31B models but nothing really larger. Go to $12-13k to upgrade to an RTX 8000 Pro and you have 96GB of VRAM, which will run larger models (but certainly not, say, DS v4 Pro or even Flash). You have shared video memory products rapidly coming from NVidia's aggressive market segmentation. Things like Strix Halo and DGX Spark have severe limits on memory bandwidth (<300GB/s compared to 1.8TB/s for a 5090/6000 Pro and 3TB/s+ for server grade HBM3e/4 based GPUs). Macs could be real interesting in this space butr they lack the raw FLOPS with the M5 generation.
But what will this local hardware look like in 2-3 years? I think people will be shocked at how much better it will be with the Apple M7 Pro/Max generation (2028 expected) and the RTX 6000 cards at that time although I fully expect NVidia consumer GPUs to still top out at 32GB of VRAM to maintain that segmentation. And I look forward to what the next generation of the AMD Ryzen AI Halo platform will look like if they really try.
All of this adds up to these three companies needing to cash out before the music stops (IMHO).
Over the last 2+ decades gambling has increased massively. By "gambling" I include sports betting, prediction markets, online casinos and crypto (including NFTs). Yes, I'm including crypto under gambling. When ordinary people buy crypto it's mostly because they saw the stratospheric rise of Bitcoin, don't really understand why and don't want to miss out on the next Bitcoin.. Straight up. That's gambling.
But why? Ease of access (ie the Intenret and particularly smartphones) are obviously a factor. There was a time when if you wanted to gamble you had to do it illegally or you had to go to a physical casino in Las Vegas or Atlantic City (and then riverboats, etc). You can look across to see how ease of access to gambling absolutely increases the amount of gambling and thus gambling addiction. A good example are the poker machines that are available in some states in Australia but not all.
But there's something deeper than that (IMHO). And I think that thing is the increasing inequality [1] and the affordability crisis. People may be talking about affordability a lot now but it's not a new issue. It was a key issue in the 2016 election, for example.
The thesis is that as people become increasingly desperate to have enough money (and espeically if they fall below that level) then gambling increases because people see this as the only way they can get ahead (eg [2]).
I believe gambling is a symptomo and indicator of a deeply broken societ and a deteriorating society.
The sad reality is that gambling is an excellent way of extracting what little wealth poor people have and it makes everything worse on a macro scale. Gambling should be relatively inaccessible. Yet we have the Federal government suing to stop states to ban prediction markets [3].
Follow-up question: who freed the serfs of 19th century Russia?
Here's another (not so) fun fact: over a million serfs were freed in 1959. "Serf" is actually generous because these serfs could be traded like property. You know, like slaves. Where was this? Tibet [1].
One of my favorite historical fun facts was that despite the Wright brothers flying the first place in 1903, at the start of WW1, the US was completely incapable of building an airplane and had to buy them from France and Great Britain. Why? Because of patent wars [1].
The net effect of that was that Congress had to intervene and they created an avionics patent pool, a system that persists to this day.
So whenever anyone says that patents foster innovation, just look at this or any number of historical counterexamples.
As for WW2, the causes were historical. The US was still suffering from the aftereffects of the Great Depression and American isolationism. It's worth noting that there was a lot of sympathy towards Nazi Germany in the US with the American Bund Party who had a rally in Madison Square Garden in 1939 [2].
I guess it's time for some jingoistic rewriting of history. If you want to sum up America's rise to power it's the slave trade, war and a healthy dose of luck (eg the Louisiana Purchase).
There is a concerted attempt to rewrite history on slavery. You will hear things like "slavery was an economic drain" or "slavery was inefficient" or even "it was technology like the cotton gin that created wealth, not slavery". All of it's nonsense [1].
It's true that industrialization (particularly the railroad ans mass production of steel) was a huge driver in the mid-19th century but what really kicked the US into high gear was war [2].
It's true that material conditions and real wages started stagnating in the 1970s but this piece writes that off as Wall Street shenanigans. This was a political goal to break organized labor. We had McKinsey producing reports to argue that executives were "underpaid" [3]. The post-war era went from a marginal tax rate of 91% and the CEO to median worker ratio went from 21:1 in 1965 to 351:1 in the 2020s [4]. But also the post-war economy shifted from housing being a utility to being a speculative asset. The median house price went from $18,000 to $26,000 between 1953 and 1973 (in nominal terms) [5] and decreased in real terms. And, well, we know what's happened since.
But what's less well-known is the link between money going into housing and decline in manufacturing. That's not an accident. Why invest money and run a factory when sitting on a house produces a 7%+ real returns that are government-protected?
As for the whole "right to repair" bit for tractors and the like, yeah, companies engage in rent-seeking behavior in a capitalist mode of production. Film at 11.
No, the hope really is these other platforms with a shared memory architecture. The DGX Spark won't be it because of the aforementioned market segmentation. So that leaves two players: AMD and Apple.
The AMD platform is still too low memory bandwidth, currently <300GB/s. For comparison a 5090 (or 6000 Pro) is 1.8TB/s and the M3 Ultra Mac Studio is ~900GB/s. Oh and B100/B200 uses HBM3e memory at ~3.2TB/s. The M5 Max in some Macbook Pros tops out at ~600GB/s. So you need access to better RAM and better CPU architecture for all this.
My great white hope is Apple. They have the market power to get memory and build silicon that coul dhave enough FLOPS to compete with NVidia's platform. They've started talking about it and I've seen rumors they're targeting the M7 generation (2028) for a huge leap. I'll believe it when I see it however.
But the point is, I think we'll be running 31B models at 100+tok/s on enthusiast hardware in 2 years and we'll likely be able to locally run 100-400B models, possibly larger.