Tokens will surely become commodities, but models don't need to.
The point is that electricity becomes a commodity, rather than, say, nuclear reactors or gas turbines also need to be commoditized.
Contrary to most people in the software-minded circle, I am not very excited about running local LLMs. Most of what I have seen involves selling wrappers that call APIs on top of an LLM AI, similar to how traditional SaaS has generated revenue from new technologies. So, this would certainly make those people who are working on that sensitive about the pricing. LLM pricing is treated as a continuous cost in the COGS. They don't really use AI to create much value (consumer surplus) other than replacing the existing boring products, like just building the good old CRUD apps, but faster.
What I find REAL interesting about the potential of LLM AI instead is to create new technology out of it, or revolutionize something old to be order-of-magnitude more efficient. In this regard, the expenses on those tokens are more akin to an upfront CapEx.
Cheaper tokens would surely be nice. But if what we are talking about is, like, solving self-driving, curing cancer, or making air-conditioners 100x efficient, the narrow focus on running a cheaper model in my home so I can write my SaaS apps for my get-rich-quick business looks really unpromising and a waste of time for human civilization in a near-singularity horizon.
Unlike the belief that frontier AI is expensive due to a high margin, and going to be expensive if there is no competition. My understanding is that, under certain circumstances (which is most likely true), the price will be driven down just because of profit seeking.
The frontier LLM labs run on a huge fixed cost and very low marginal cost. They need the economies of scale to make sense of the business (an incentive to expand their user base as large as possible). Imagine that you want to buy a few B300s to run GLM 5.2 and rent the service out to other people. How could this business be viable and sustainable in the first place? You need as many customers as possible. If you charge everyone $1000, you find fewer customers who can afford it. It rots the ROA if the servers are not utilized 100% (you would better buy less compute instead).
Also, the marginal cost for onboarding a new customer is low. And it's getting even lower when you have more customers. You wouldn't leave money on the table (especially for your competitors) if you want to maximize your profit.
By this logic, all frontier AI labs are incentivized to lower the price to maximize their customer base, profit, and ROA.
I don't understand how knowledge, either public or private, could get damaged.
Though the income of the individuals and businesses that rely on the expertise of the knowledge would be damaged. Is that what you meant?
Edit:
At this stage, the revenue made by the AI labs is almost entirely spent on the formation of fixed capital and opex. The demand is mobilizing physical resources with money. Atoms are relocated and reconfigured into compute racks. But eventually, the created productivity will perhaps make supply-elastic goods extremely cheap and abundant, while the supply-inelastic goods will be worth even more relative to the elastic ones.
After all, money is simply a token for the transmission of physical resources. It doesn't create stuff out of thin air. When new stuff is created, it just makes money cheaper, so that the banks can respond with more money to counteract it. More stuff -> deflation -> more money creation allowed to undo the deflation.
But the "exchange rates" between different goods and services will diverge. That's also why I don't think a direct money transfer like UBI would fix the problem, when it doesn't change the divergence of relative economic values of different goods. Let's say, extremely cheap software and entertainment, but unaffordable healthcare and housing. More money for everyone doesn't make limited resources available. So, what I am leaning into is some sort of Georgist policy. That could hopefully mitigate the price divergence, assuming that AI cannot make every commodity equally abundant.
We cannot always want to capture only the (temporary) winners whenever we see a lucrative business and expect to share a free ride. I'd also assume that most of the revenue these AI labs are making is turned into depreciating fixed capital (hardware) and OPEX at this point.
Why don't we capture Meta and Google as they allegedly take advantage of more publicly available information for profit? Let alone the truly valuable knowledge, like mathematics, has nothing to do with the majority of garbage posts that an average person would "contribute" on social media.
If we really want to tax or nationalize some economic activity, then, in my opinion, the target should be what it takes from society, not what it produces for society. By this logic, we should tax all labs, including those lagging ones, that utilize the public knowledge.
However, if everyone can access the public knowledge without rendering it less useful or reducing its available quantity, there should be no reason to tax it.
It's always funny to see people tempted to call open-blobs/open-weights, which are literally shareware like WinRAR or Adobe PDF Viewer, open source, and then need to invent a new term for what is actually open source.
Similarly, some bosses might believe that they can hire 100 cheap, unmotivated SWEs to replace Linus Torvalds or Fabrice Bellard and achieve something slightly worse. But in certain areas, it doesn't work like that.
It takes billions of investments for infrastructure, and a high-paying, top-notch team for R&D and operations. Not just a bunch of torrents of pirated books. Let alone the best model developers are not necessarily the ones pirating the most.
It's funny that Google, Meta, TikTok, OnlyFans, PornHub, and many other lucrative businesses never open-source their core business software, and people just don't bother about it with that moral standard, simply because we don't need to pay for the service (paid by ads, actually). To me, that is the hypocrisy.
I don't think that learning from textbooks to take an exam and learning from the answers of another student taking the exam are the same.
Joking aside, I also don't believe that maximum access to raw Internet data and its quantity is why some models are doing better than Google. It seems that these SoTA models gain more power from synthetic data and how they discard garbage.
6% compared to the post-2000s is mediocre, especially given the low baseline. Not remarkably better than other high-income democratic countries like Japan and West Germany. Even the US can have ~4% growth at the time.
American business leaders have (had?) an obsession with gross margin and tech "advancedness." They thought they would be the winner as long as they occupied the high-tech sectors in the supply chain. So they discarded the high-volume, low-margin, low-growth, low-tech businesses like assembly lines and outsourced them. But the reality is that the proximity of the assembly lines creates a cost advantage that attracts more upstream suppliers to surround it. Even Intel was seeking to build more fabs in China before being stopped by the US government.
If that were the true secret sauce of the economic success in China, why had it not taken off before the 2000s? Like, they have been that "aligned" and "want the same thing" and "run by engineers" since the 50s, no?
The silicon shield became a slogan that has only been popularized in recent years. The potential crisis of war has been there for more than half a century (even before semiconductors became a thing). The real value proposition of the status quo is the freedom of navigation between the northeastern Asian countries and the SEA (the Strait of Malacca, aka the lifeline of energy imports), and the consequential domino effect of the entire western Pacific.
Also, not sure why everyone forgets about it. People should have learned from the experience of the pandemic that the cutting-edge foundry nodes are not really the crucial ones, as being the bottleneck of industrial infrastructure. A delay of the next-gen iPhone or RTX gaming card isn't that catastrophic. But a shortage of embedded MCUs, which are actually fabricated by mature nodes, could stall the entire industrial base of a country.
I'd bet, on average, the quality of proprietary code is worse than open-source code. There have been decades of accumulated slop generated by human agents with wildly varied skill levels, all vibe-coded by ruthless, incompetent corporate bosses.
This is only relevant to the compute productivity (how much useful work it can produce), but it's irrelevant to the heat dissipation problem. The energy income is fundamentally limited by the solar facing area (x 1361 W/m^2). So the energy output cannot exceed it, regardless useful signals or just waste heat. Even if we just put a stone there, the equilibrium temperature wouldn't be any better or worse.
That should be better than a sphere. Though I imagine there could be some fancier 3D geometry designs.
Even for a simple sphere, if we give it different surface roughnesses on the sun-facing side and the "night" side, it can have dramatically different emissivity.
Assuming that we place an iron ball (ideal sphere-shaped and thermal conductivity) on the SSO (solar synchronous orbit), how hot can the object be?
Given the solar constant 1361 W/m^2, you can calculate the temperature range based on the emissivity and absorptivity. With the right shape and “color”, the equilibrium temperature can be cooler than most people thought.
I suppose that a space data center powered 100% by solar is no different than this iron ball in principle.
The radiator area is probably not what they need to worry about that much as we thought. When the energy input comes from solar 100%, they just need to optimize the ratio of the sectional area facing the sun over the total surface area of the satellite. If the ratio is low enough, like a fin or cone shaped object, it will be harder to be hot.
The point is that electricity becomes a commodity, rather than, say, nuclear reactors or gas turbines also need to be commoditized.
Contrary to most people in the software-minded circle, I am not very excited about running local LLMs. Most of what I have seen involves selling wrappers that call APIs on top of an LLM AI, similar to how traditional SaaS has generated revenue from new technologies. So, this would certainly make those people who are working on that sensitive about the pricing. LLM pricing is treated as a continuous cost in the COGS. They don't really use AI to create much value (consumer surplus) other than replacing the existing boring products, like just building the good old CRUD apps, but faster.
What I find REAL interesting about the potential of LLM AI instead is to create new technology out of it, or revolutionize something old to be order-of-magnitude more efficient. In this regard, the expenses on those tokens are more akin to an upfront CapEx.
Cheaper tokens would surely be nice. But if what we are talking about is, like, solving self-driving, curing cancer, or making air-conditioners 100x efficient, the narrow focus on running a cheaper model in my home so I can write my SaaS apps for my get-rich-quick business looks really unpromising and a waste of time for human civilization in a near-singularity horizon.