Inkling: Our Open-Weights Model(thinkingmachines.ai)
thinkingmachines.ai
Inkling: Our Open-Weights Model
https://thinkingmachines.ai/news/introducing-inkling/
316 comments
Not to mention - it is American. This is the first competitive non-Chinese open weights model since what, Llama 3?
North Mini Code by Cohere (HQd in Toronto) has honestly been very competitive in my personal assessment with many of the models coming out of the PRC. I'd position it below Moonshot AIs and Z.ais recent releases, but above the varieties of Qwen, Deepseek, MiMo, etc.
Depends whether America the continent or just the United States counts of course.
Depends whether America the continent or just the United States counts of course.
>Cohere (HQd in Toronto)
Cohere keeps changing their story about where they're headquartered. From 2019-2020 they were HQed in Toronto. Then from early 2020 to 2026 they billed themselves as dual-headquartered in Toronto and San Francisco. Then in April 2026 they started to bill themselves as dual headquartered in Toronto and Berlin. Sometime in the last two months they've started to bill themselves as HQed in Toronto again.
Cohere keeps changing their story about where they're headquartered. From 2019-2020 they were HQed in Toronto. Then from early 2020 to 2026 they billed themselves as dual-headquartered in Toronto and San Francisco. Then in April 2026 they started to bill themselves as dual headquartered in Toronto and Berlin. Sometime in the last two months they've started to bill themselves as HQed in Toronto again.
Interesting, I didn’t know this existed, do you think it’s competitive with Qwen 3.6 35B A3B which seems to be the closest comparator?
It’s 20 vs 32 in favor of Qwen on artificial analysis intelligence index (cohere isn’t benchmarked on the coding index)
It’s 20 vs 32 in favor of Qwen on artificial analysis intelligence index (cohere isn’t benchmarked on the coding index)
It is surprisingly capable for coding despite its tiny size.
You can try it out for free:
https://opencode.ai/docs/zen/#pricing
You can try it out for free:
https://opencode.ai/docs/zen/#pricing
It's not the better model since Llama3. Trinity Large is American and quite decent, unfortunately tons of crazy good models have been out and it's harder to run locally at 400B. I think Arcee did a terrible job of promoting their model.
https://www.arcee.ai/blog/trinity-large
https://www.arcee.ai/blog/trinity-large
+1 I enthusiastically use Chinese open weight models for a wide range of tasks (I also love Opus and Gemini) but I am so happy to see another high quality American open model (I consider gemma to be high quality, like qwen).
I enjoyed turning off web search for Inkling to experiment with what innate knowledge is encoded in the model weights. A fun thing I do is check what innate knowledge very large models contain about me, as an individual. Inkling has an interesting concise shadow of what I do. (I have written a lot of books, so I am in training data.)
I enjoyed turning off web search for Inkling to experiment with what innate knowledge is encoded in the model weights. A fun thing I do is check what innate knowledge very large models contain about me, as an individual. Inkling has an interesting concise shadow of what I do. (I have written a lot of books, so I am in training data.)
What does it mean, if it is American?
Is it censored or will it eventually stop working in Middle Eastern countries?
Or is it biased towards powerful political lobby group interests?
When weights are open I usually don't care where is it from, as long as it is working for my use cases well
Is it censored or will it eventually stop working in Middle Eastern countries?
Or is it biased towards powerful political lobby group interests?
When weights are open I usually don't care where is it from, as long as it is working for my use cases well
There's a lot of Sinophobia in the tech world, and the AI race seems to be magnifying that. With very little evidence to back it up, I might add - we've had regular releases of quality open-weight models from Chinese firms, and no sign they are more censored/ideological, than, say, Grok
The Chinese are quite open about their efforts regarding intellectual espionage and the desire for control of other countries' supply chains. It's not sinophobia to be cautious about it. It doesn't mean that the US companies are angels either.
Only actualy censored thing i found so far was Tianmen Square. And depending on what you use it for thats realy not a big issue.
Any discussion about the Chinese Communist Party is censored on Deepseek.
If the company does any kind of work with the US Government it's far easier to just say no to Chinese-origin models rather than deal with all the overhead on the government contracts. Since just about every major tech company has US contracts they all have to be careful about the use of Chinese models.
Chinese models are much more censored than US ones. Just ask a broad question such as "Please criticize the chinese communist party". Besides, Grok has been proven more balanced than OpenAI/Anthropic's models.
There is also NVIDIA Nemotron 3 Ultra, with 561B parameters, which was released a month ago.
I do not know yet how smart it is, but the NVIDIA LLMs are very well optimized for fast inference (on their GPUs of course).
Previously that was the biggest American open-weights LLM.
I do not know yet how smart it is, but the NVIDIA LLMs are very well optimized for fast inference (on their GPUs of course).
Previously that was the biggest American open-weights LLM.
Gemma 4
The largest Gemma 4 model has 31B parameters. It’s not in the same class.
I think there's two halves to the conversation: which models have more weights and which models are better than the other ones listed. I think this was about the latter part. There are plenty of smaller models these days which knock the socks off older models 10x the size.
We are talking about the top tier of open weights models - GLM-5.2, DeepSeek V4, Qwen3, Kimi K2. The ones ranking on leaderboards and giving frontier labs a run for their money. Gemma may have its uses but it is not in that conversation.
I find it easy enough to interpret as talking to "a better non-Chinese model since Llama 3", even if it doesn't compete with the current large Chinese models mentioned before that. Again, they aren't saying you're wrong about large models - they are adding that Llama 3 was separately surpassed by smaller non-Chinese models during the wait.
Isn't that British?
British-American with much of the research happening in London. I don't know if it's known what team specifically worked on which of the Gemmas, I'd suspect it's a healthy mix of multiple satellites across the globe, but the contribution by the UK Deepmind parts likely was significant, certainly not so small that it'd justify downing a regular question.
"Competitive" is doing a lot of work there — it debuts at #41 on the AA index and the post itself calls it not the strongest overall. Mistral and Cohere's North are also non-Chinese open weights, so Llama 3 wasn't the last.
GPT OSS was post Llama 3 and pretty strong at the time. But yeah this is the first seriously competitive non-Chinese open model in a good bit now.
There's also poolside.ai
I was blown away by how effective their latest model drop that works with their own coding harness pool is. I tested it extensively with Haskell, Python, and TypeScript for small coding projects. The functionality is good but the inference speed is too slow for most of my work: I would set up a problem, take a walk, then return later to evaluate the results. Note that I have an old mac mini with 32B memory; a fast modern home system would be much better.
EDIT: their business model is interesting, aiming for supporting organizations with privacy and security concerns.
EDIT: their business model is interesting, aiming for supporting organizations with privacy and security concerns.
Yeah, they’re focused on selling to government and defense contractors. I first encountered poolside at my aerospace internship.
That is a nice business niche, but I would hope that companies might also care about security. There are just a few inference providers who I trust to not save data outside the scope of maintaining inference cache, but I do use untrusted providers sine everything I work on is open source or open content.
I was hoping for some relation to poolside.fm but alas. This looks cool though.
Or rather, Albanian.
nemotron
> it is American
It's also–hopefully–run by a cooler head. Altman launching a nuke into his own backside with his "stop me before I shoot grandma" routine was at least novel. Dario repeating the same playbook to the same effect years later still genuinely confounds me.
If Thinking Machines pans out I could see it finding a welcome home at Apple.
It's also–hopefully–run by a cooler head. Altman launching a nuke into his own backside with his "stop me before I shoot grandma" routine was at least novel. Dario repeating the same playbook to the same effect years later still genuinely confounds me.
If Thinking Machines pans out I could see it finding a welcome home at Apple.
> If Thinking Machines pans out I could see it finding a welcome home at Apple.
I was thinking the same thing. Apple's use of Gemini can't be a long-term solution.
I was thinking the same thing. Apple's use of Gemini can't be a long-term solution.
Llama 4 was unfairly hated on. Still the longest context window on any LLM ever (so what if you can't use it properly?) and unironically had decent image capabilities compared to llama3 which had none.
Benchmark cheating aside, it wasn't that bad.
Benchmark cheating aside, it wasn't that bad.
I mean if you don't care it's utilized properly you can make a lot of local models have > 10 million context length. I don't know why you would, the quality is already crappy enough when models are built around using it well from the ground up, but go ahead.
The hate was fully deserved considering Meta introduced it with deliberately manipulated benchmarks.
gpt oss was competitive when it cam out
> This supposedly is better than KimiK2.7
How can you tell?
I just looked at the benchmarks and was kinda disappointed that it seems to be between KimiK2.6 and KimiK2.7 on most of the benchmarks.
Do you refer to what it feels like to use the model? Or are there other benchmarks I haven't seen?
How can you tell?
I just looked at the benchmarks and was kinda disappointed that it seems to be between KimiK2.6 and KimiK2.7 on most of the benchmarks.
Do you refer to what it feels like to use the model? Or are there other benchmarks I haven't seen?
Most of the random comments you read on HN and reddit about how nice/bad various LLMs are, is basically based on the commentator's "vibe" about it, and almost nothing is grounded in evidence or actual usage. Don't read too much into it, want to know how good a model is? Run it with your own non-public benchmark, basically the only way to get proper answers you can somewhat rely on, everything else is manipulated, misunderstood or over-relied on.
I thought HN was different. And yeah, wherever I go, my timeline is full of Opus is so bad today and I will switch from Fable to 5.6 Sol, it's 1.5x better and vice versa.
Non-public benchmarks (ideally suited to one's own use case) are probably the best way to judge, I agree.
Non-public benchmarks (ideally suited to one's own use case) are probably the best way to judge, I agree.
I also have been preferring kimik2.7 more than GLM5.2. I'm interested to give this a try.
Where are you getting supposedly. It does worse in most benchmarks
Oh thanks for sharing! The llama.cpp PRs should generally be fine for now - I'm fixing a few small edge cases as well!
I'm sure it's better than KimiK2.7 and GLM5.2. Benchmarks aren't the full picture. Despite GLM5.2 performing well on benches and supposedly near frontier, in reality it was nothing close to frontier in actual usage.
Have you used Inkling enough to be able to tell? Or how can you be sure? Please add some substance before making such claims
America needs its own DeepSeek or Z.ai, a lot of people (myself included) root for open chinese models to win because they have no other choice.
Thinking Machines might be it.
Thinking Machines might be it.
I don't hear about them a lot but it looks like arcee.ai is aiming to be just that.
Here are some of their current open weight offerings: https://www.arcee.ai/open-source-catalog
Here are some of their current open weight offerings: https://www.arcee.ai/open-source-catalog
You don't hear about them much because their models aren't really competitive. I really wanted to try Trinity Large as a daily-driver in the MiniMax M2 sort of niche but I couldn't make it through a single day. The models need another couple point releases worth of post-training to make useful agents and if memory serves they weren't any less slopped in writing style and those are really the only two things people look for in models.
> root for open chinese models to win
What does "winning" mean to you?
What does "winning" mean to you?
It could be but there are a host of companies going after open weights models: Arcee, Reflection, Llama (TBD on Meta's focus on closed-source versus open-source), etc.
That said, the fine-tuning API + open weight model at least is a semblance of a viable business that could work so I will be curious about it. I'm not sure the synergy is fully there (why is someone with an open weights model privelaged to fine-tune it better if it's just QLora or Lora) but let's see!
That said, the fine-tuning API + open weight model at least is a semblance of a viable business that could work so I will be curious about it. I'm not sure the synergy is fully there (why is someone with an open weights model privelaged to fine-tune it better if it's just QLora or Lora) but let's see!
I don’t really get the business plan for open weights model companies, is the idea companies would pay them for serving?
Thinky's main commercial product AFAIK is Tinker [0] - companies pay them to host their fine-tuning workloads and then the resulting fine-tuned models. I don't know if this is a good business plan, but I'm sure at least one person there has read Joel on Software [1].
[0] https://thinkingmachines.ai/tinker/
[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
[0] https://thinkingmachines.ai/tinker/
[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
I don’t know if it’s a great business model but it makes perfect sense to me. Open models when fine tuned are capable at better than frontier performance at a fraction of the price for many (probably most) domain specific tasks. If companies help make that easy to implement, there is value to capture. But I kind of like Unsloths model here which is to be really good at just layer, and not bothering with building their own models.
I don't get this - I can do LORA on my mac... ok I can't do LORA on a 1tn param model, but if I was in the tn parameter model game I would get some kit that I could use to do that...
What's their moat / secret sauce?
What's their moat / secret sauce?
Like, buy and set up the physical hardware? I cba with that. Plus the hardware you want for LoRA (the type but especially the quantity) is different than what you want for inference, so either you'd under-spec it and wait forever for fine-tuning runs, or over-spec it and have low utilization most of the time. And even then who knows if it would be good enough to LoRA next year's best open source model. AWS gets great margins for renting out commodity hardware as a service because it built the right abstractions and can serve them efficiently at scale, I think the arguments here are basically the same.
From what I can tell it’s a bit of the following.
1. Magic
2. Managed hosting of their model
3. Hurting competitors. If people are using Meta’s commoditized models they’re not paying Google or allowing OpenAI to become too big.
4. Free R&D from open source. If open source developers are optimizing systems to run Llama, that helps Meta.
5. More magic
1. Magic
2. Managed hosting of their model
3. Hurting competitors. If people are using Meta’s commoditized models they’re not paying Google or allowing OpenAI to become too big.
4. Free R&D from open source. If open source developers are optimizing systems to run Llama, that helps Meta.
5. More magic
I think that LeCunns belief was/is that LLM's have limited value and are a dead end, so what they wanted to do was kill any competitor while evolving tech that was/is a winner.
Well, he lost his job on that bet... and yet... I do not think that the verdict of history is quite in.
Well, he lost his job on that bet... and yet... I do not think that the verdict of history is quite in.
> don’t really get the business plan for open weights model companies
Feature as a company for now. Apple is struggling to build an in-house model set. And plenty of software behemoths, e.g. IBM, are realising they don't have a ticket to the new tech economy.
Feature as a company for now. Apple is struggling to build an in-house model set. And plenty of software behemoths, e.g. IBM, are realising they don't have a ticket to the new tech economy.
> don't really get the business plan for open weights model companies
The Chinese "Neijuan" aside, most competing labs are going for the classic, 'your margin is my opportunity': https://tomtunguz.com/is-your-margin-my-opportunity-software... / https://archive.vn/5Vmq3
The Chinese "Neijuan" aside, most competing labs are going for the classic, 'your margin is my opportunity': https://tomtunguz.com/is-your-margin-my-opportunity-software... / https://archive.vn/5Vmq3
Llama is dead. Meta is now releasing proprietary models (Muse Spark).
They’ve made some wishy-washy statements about their intention to release a future version of Muse Spark as open weights. We’ll see.
Do any of these even have match a year old Deepseek 3.1?
DS3 isn't even looked at anymore.
GLM-5.2 is the best in that class right now. It is competitive with current GPT/Claude/Gemini.
GLM-5.2 is the best in that class right now. It is competitive with current GPT/Claude/Gemini.
"Current GPT/Claude/Gemini" is not a meaningful statement about perf. There's many different models from each of those providers and there's a considerable gap between the best of anthropic and open ai compared to gemini.
Benchmarks have GLM 5.2 somewhere underneath Sol and Fable and closer to now last-gen openai and anthropic models.
Benchmarks have GLM 5.2 somewhere underneath Sol and Fable and closer to now last-gen openai and anthropic models.
My previous comment from 24 hours ago is now irrelevant.
Newly released Kimi K3 is benching better than Claude Opus 4.8. The only better models are Claude Fable and GPT 5.6 Sol Max Effort.
Newly released Kimi K3 is benching better than Claude Opus 4.8. The only better models are Claude Fable and GPT 5.6 Sol Max Effort.
One error: GLM 5.2 beats the best public Gemini model, 3.5 pro.
There's 2 caveats with the rest. First, GLM 5.2 matches those models in "xhigh" effort modes, which has a very low quota on the subscriptions, especially for Claude.
Second, last-gen GPT/Claude means what they release in April/May of 2026. Or to be even more complete/fair:
GLM 5.2 beats what OpenAI released in March 2026 (GPT 5.5 xxhigh), and what Anthropic released in April 2026 (Opus 4.7 xhigh). It is beaten by what OpenAI released in April of 2026 (GPT 5.6 Sol xxhigh) and Anthropic released in May 2026 (Opus 4.8 (the same as "Fable" ?), xhigh effort)
GLM 5.2 was released on Jun 16 and if OpenAI and Anthropic hadn't done those quick releases they would have been beaten on their best available models ...
So great news! Open source now has SOTA performance 3 months after OpenAI/Anthropic/Google. Wow.
There's 2 caveats with the rest. First, GLM 5.2 matches those models in "xhigh" effort modes, which has a very low quota on the subscriptions, especially for Claude.
Second, last-gen GPT/Claude means what they release in April/May of 2026. Or to be even more complete/fair:
GLM 5.2 beats what OpenAI released in March 2026 (GPT 5.5 xxhigh), and what Anthropic released in April 2026 (Opus 4.7 xhigh). It is beaten by what OpenAI released in April of 2026 (GPT 5.6 Sol xxhigh) and Anthropic released in May 2026 (Opus 4.8 (the same as "Fable" ?), xhigh effort)
GLM 5.2 was released on Jun 16 and if OpenAI and Anthropic hadn't done those quick releases they would have been beaten on their best available models ...
So great news! Open source now has SOTA performance 3 months after OpenAI/Anthropic/Google. Wow.
The gap has been steadily closing over time.
Opus 4.8 (May) to Kimi K3 (July) has apparently just dropped it to two months.
China also does efficiency improvements. Qwen 3.6 27B is better than Sonnet 4.5 and you can run it on a couple of gaming video cards. That's incredible. I can do real actual work with this!
As Google said in 2023, none of them have a moat, open weight models will win.
Opus 4.8 (May) to Kimi K3 (July) has apparently just dropped it to two months.
China also does efficiency improvements. Qwen 3.6 27B is better than Sonnet 4.5 and you can run it on a couple of gaming video cards. That's incredible. I can do real actual work with this!
As Google said in 2023, none of them have a moat, open weight models will win.
> It could be but there are a host of companies going after open weights models: Arcee, Reflection, Llama (TBD on Meta's focus on closed-source versus open-source), etc.
my bet is that Chinese government fund Chinese models way more compared to what those companies receive (except llama, which is outdated but was strong foundation at its time)
my bet is that Chinese government fund Chinese models way more compared to what those companies receive (except llama, which is outdated but was strong foundation at its time)
The story of Reflection AI is supposedly that the company was faffing and failing at winning in the coding agent space, but was introduced to Jenson, who suggested they build an open-weight model and said he would fund it. That turned into a $2 billion financing with NVIDIA doing roughly $500 million and was a complete pivot.
I think the bet would have to be that a US Open Weight company either: 1. Gets a lot of money from Jenson who views them as a counterbalance to the big labs in his ecosystem and a way to generate leverage (the same way he is positioning neoclouds-- it also could be synergistic with neoclouds who could offer the model serving endpoints) 2. Can fast follow the same way Mistral does (which, honestly, seems like just distilling the Chinese model, which distills the US lab but is pretty innovative on a whole lot of architecture both in training and serving land.) 3. AND figure out some (maybe not super lucrative but lucrative enough) sort of business model, as well. There are lots of possible business models, so I will be curious how this whole space evolves.
I think the bet would have to be that a US Open Weight company either: 1. Gets a lot of money from Jenson who views them as a counterbalance to the big labs in his ecosystem and a way to generate leverage (the same way he is positioning neoclouds-- it also could be synergistic with neoclouds who could offer the model serving endpoints) 2. Can fast follow the same way Mistral does (which, honestly, seems like just distilling the Chinese model, which distills the US lab but is pretty innovative on a whole lot of architecture both in training and serving land.) 3. AND figure out some (maybe not super lucrative but lucrative enough) sort of business model, as well. There are lots of possible business models, so I will be curious how this whole space evolves.
Jensen Huang is just trying to commoditize the complements to his GPUs.
> That turned into a $2 billion financing with NVIDIA doing roughly $500 million and was a complete pivot.
I suspect 2B is not enough to boostrap frontier model from the scratch (for both talent and hardware)
I suspect 2B is not enough to boostrap frontier model from the scratch (for both talent and hardware)
>The story of Reflection AI is supposedly that the company was faffing and failing at winning in the coding agent space, but was introduced to Jenson, who suggested they build an open-weight model and said he would fund it. That turned into a $2 billion financing with NVIDIA doing roughly $500 million and was a complete pivot.
You can pretty much remove the supposedly here
You can pretty much remove the supposedly here
AllenAI is also one to keep your eye on. Founded by Paul Allen of Microsoft, they are one of the best teams working towards truly transparent / open AI (including training data)
I love the tasteful thickness of Paul Allen’s model.
I love Allen AI.
I find it wonderful that, as a non-profit, they are only one to two years behind SOTA models that cost billions of dollars to build, if not more.
I find it wonderful that, as a non-profit, they are only one to two years behind SOTA models that cost billions of dollars to build, if not more.
AllenAI is great, but they don't have the budget or remit to build large models.
I wonder if the recent sale of the Seahawks will change that. IIRC, ~$10B and all is supposed to go to charity. Not sure how much of that will go to AllenAI, though. (If any.)
No reason to think it will. Paul Allen died after donating the money to create AllenAI and I don't think there are any links.
Hopefully it somehow works out though!
Hopefully it somehow works out though!
I will wait for Modernist AI by Myhrvold.
What is the business model for an open weight model?
The same business model that Deepseek is using.
Open-source models + services. This is more attractive because it doesn't lock in the vendors. If I grow larger, I can decide to deploy the open-source models.
Open-source models + services. This is more attractive because it doesn't lock in the vendors. If I grow larger, I can decide to deploy the open-source models.
So they're constantly hemorrhaging their most valuable clients?
Tech history is littered with the corpses of "open source but we sell hosting" services. Models are so expensive to train, you can't be losing the big clients once they get super profitable.
Tech history is littered with the corpses of "open source but we sell hosting" services. Models are so expensive to train, you can't be losing the big clients once they get super profitable.
This is genuine, noob question: how is this different from AWS?
I get that they're in very different businesses, but for both don't they have the issue that once a client gets big enough the client might decide to move the services in-house? Based on how much of the internet went down when that AWS data center crashed the answer is clearly "No" for AWS.
Is that because of physical, real-world infrastructure? Are there no open versions of their APIs? Is it too hard to migrate to something else once a client has achieved that size?
I get that they're in very different businesses, but for both don't they have the issue that once a client gets big enough the client might decide to move the services in-house? Based on how much of the internet went down when that AWS data center crashed the answer is clearly "No" for AWS.
Is that because of physical, real-world infrastructure? Are there no open versions of their APIs? Is it too hard to migrate to something else once a client has achieved that size?
Data is heavy.
I would say "it's risky and requires a lot of labor to migrate without corruption, loss of data" and also minimizing downtime. Sure anyone can run pg_backup, but can you do it across 90 databases? Can you do it live? Can you coordinate rollout of the process, cutover, and monitor for failure? What's the cost of egress for this? Is the team your A-team or the B-team? Can you trust this to the B-team? Is it worth having this team spend all this time on a migration rather than, say, getting something new set up, or optimizing performance on an existing system?
I'm a database guy, but the same migration argument is presumably also extra work for (say) blob storage, networking, etc.
Since LLMs are stateless by their current implementation, switching to "the same open-weight model running in a different datacenter run by a different vendor" is "just" switching the API endpoint. (If they are the exact same shape, it's fine, if they differ somehow, there's perhaps some work to do there, fixing things and monitoring for failures on switch-over)
There are several open APIs it seems and OpenRouter.ai is doing a fine job making a commodity out of models and datacenters.
I would say "it's risky and requires a lot of labor to migrate without corruption, loss of data" and also minimizing downtime. Sure anyone can run pg_backup, but can you do it across 90 databases? Can you do it live? Can you coordinate rollout of the process, cutover, and monitor for failure? What's the cost of egress for this? Is the team your A-team or the B-team? Can you trust this to the B-team? Is it worth having this team spend all this time on a migration rather than, say, getting something new set up, or optimizing performance on an existing system?
I'm a database guy, but the same migration argument is presumably also extra work for (say) blob storage, networking, etc.
Since LLMs are stateless by their current implementation, switching to "the same open-weight model running in a different datacenter run by a different vendor" is "just" switching the API endpoint. (If they are the exact same shape, it's fine, if they differ somehow, there's perhaps some work to do there, fixing things and monitoring for failures on switch-over)
There are several open APIs it seems and OpenRouter.ai is doing a fine job making a commodity out of models and datacenters.
I don't think it's that difficult. Their servers are stateless too. S3 is easy to migrate.
Database is more difficult, but tons of people have done it successfully.... meanwhile people who host their own LLMs are relatively small in number in comparison.
Most companies don't do their own data centers mainly because it is more expensive and less reliable. It's something they can just pay for the problem to go away. The calculus for hosting your own LLM is probably similar.
Even Stripe who built their own coding agents and has tons of money/resources still decides not to host their own LLMs.
Still, many people will prefer open-weight models. It is similar to how we prefer linux but still use AWS/Render/and whatever. It doesn't lock us in, and we can move providers if we want to.
Database is more difficult, but tons of people have done it successfully.... meanwhile people who host their own LLMs are relatively small in number in comparison.
Most companies don't do their own data centers mainly because it is more expensive and less reliable. It's something they can just pay for the problem to go away. The calculus for hosting your own LLM is probably similar.
Even Stripe who built their own coding agents and has tons of money/resources still decides not to host their own LLMs.
Still, many people will prefer open-weight models. It is similar to how we prefer linux but still use AWS/Render/and whatever. It doesn't lock us in, and we can move providers if we want to.
> This is genuine, noob question: how is this different from AWS?
AWS owns the hardware, and doesn't write a lot of the software.
AWS actually is kind of the opposite - it often takes open source software (e.g. Apache, Mongo, Kubernetes) and then makes money off it by hosting it itself (with some enhancements etc).
If they do develop their own software (e.g. with S3) they don't give away the source code so others can deploy it, as that's part of their secret sauce.
In this scenario, where they would be offering the open source model and then offering the same model hosted, there isn't really a moat here - they would be leasing the hardware from a company like AWS, and adding a margin, but it woudl be trivial for another company (or Amazon) to take their same model and offer it for the same price or less.
AWS owns the hardware, and doesn't write a lot of the software.
AWS actually is kind of the opposite - it often takes open source software (e.g. Apache, Mongo, Kubernetes) and then makes money off it by hosting it itself (with some enhancements etc).
If they do develop their own software (e.g. with S3) they don't give away the source code so others can deploy it, as that's part of their secret sauce.
In this scenario, where they would be offering the open source model and then offering the same model hosted, there isn't really a moat here - they would be leasing the hardware from a company like AWS, and adding a margin, but it woudl be trivial for another company (or Amazon) to take their same model and offer it for the same price or less.
> The same business model that Deepseek is using.
there is a chance their business model is absorbing government funding..
there is a chance their business model is absorbing government funding..
Better than average chance I’d say. I suspect they are hoovering up EVERYTHING that gets sent to them. Whether that’s a problem or not depends on your data. I do wonder how many security tokens they get in the stream on a daily basis.
You don't have to assume that. They just IPO'd
If they get subsudized say hardware, it won't be captured in filings.
I don't think it's bad if the mandate of that funding is to have open-weights.
its not representative example of open weight business sustainability.
To compete against America. If your country has something like DeepSeek you really can't afford to let it fall as it's your best leverage if the US government decides to ban companies in your country from accessing American LLMs. And this is why there will never be a "DeepSeek of the US."
Considering how volatile things can get depending on who's president, I'd say even American companies need to "compete against America" if they don't want to get their rug pulled from under them (which, apparently, the legal system allows to easily happen in the US).
Not to mention the entire world outside the US and China. China seems to have the edge in stability.
Thinky has a potential answer in Tinker — give away the weights and charge for the SFT (and maybe RL down the line) to make the model more capable for specific tasks.
SFT/RL can be done without parent company.
But not conveniently. This is why you outsource to vendors. Not everyone will do it
In the US, there isn't one, which is why nobody in the US is currently doing it at frontier scale. And the people that were doing it stopped.
[deleted]
I’m trying to be charitable but your comment reads as “China bad” propaganda to me. Who cares that DeepSeek and Z.ai are Chinese companies?
It doesn't matter until it does. If the chinese government decides that open weight model releases are no longer allowed, that's a lot of companies that can't release new models. Same with the US government, etc. Having diversity is important.
However unlike the US models, China banning the release of new models would not break existing ones. Betting on US models only can get you locked out in just a few hours.
No, Open weights US models would not break as well - this isn't related to China or USA, it's about Open Weights and the fact that you can download the models.
It's a similar problem the human DNA solved by telling our teenage selves that our parents are dumb and we needed to move to a new tribe. Genetic diversity, but a digital equivalent.
China’s got absolute control over its outputs. For America to have any guarantees around long-term availability of OW models, they need domestic production.
FWIW this is the same logic for China’s need for their own OW models
FWIW this is the same logic for China’s need for their own OW models
LLMs aren't just for coding and math. Many people understand the world through LLMs, even when it comes to philosophy and politics.
If you understand the world through a Chinese LLM, you are seeing it through a biased lens stemming from biased training data.
(Also, in that way, having all major LLMs developed by the US carries a risk too. We need more diversity than just the viewpoints of the US or China.)
If you understand the world through a Chinese LLM, you are seeing it through a biased lens stemming from biased training data.
(Also, in that way, having all major LLMs developed by the US carries a risk too. We need more diversity than just the viewpoints of the US or China.)
All 6 UN languages should have their own dedicated LLM at the very least: American for English, Chinese for Chinese, LATAMgpt for Spanish, Russia has their own, Mistral for French, the Arabs don't have one yet I think, then there's sarvam for India and South Korea's sovereign models like SK telecom's
I think practically every government will want to put restrictions on private companies building models.
Frankly the EU and the US will practically be less involved and have more pushback from the public in this than China. I think that’s less “China bad” than recognizing that China is a more authoritarian state and has far more proclivity to interfere than western states.
Maybe I’m wrong? What does deep seek say about Tiananmen square in 1989?
Frankly the EU and the US will practically be less involved and have more pushback from the public in this than China. I think that’s less “China bad” than recognizing that China is a more authoritarian state and has far more proclivity to interfere than western states.
Maybe I’m wrong? What does deep seek say about Tiananmen square in 1989?
https://chat.deepseek.com/share/uf8jih4q95lbq3s8t2
This is what Deepseek replied when I asked it with a burner account. Claims it doesn't have it in its training data... sure.
This is what Deepseek replied when I asked it with a burner account. Claims it doesn't have it in its training data... sure.
Be interesting to see if it's possible to jailbreak and have it talk about it.
I am sure it can be but like it does prove the point of state interference in what the LLM is allowed to say/do in this case.
Ask an American model about the Epstein files and trump's involvement of them. Or about Israel committing genocide in Palestine.
Every model will have their own bias. Freedom is relative and American freedom is not the only form of freedom. China bad or China authoritarian and thus not free is a result of the red scare
Every model will have their own bias. Freedom is relative and American freedom is not the only form of freedom. China bad or China authoritarian and thus not free is a result of the red scare
Here is what Sonnet says on genocide:
"Surveys of genocide scholars (e.g., one by the journal Journal of Genocide Research contributors) show meaningful disagreement, though a visible and growing share of specialists in genocide studies specifically have concluded the term applies or is defensible — more so than international law scholars generally, who tend to be more cautious about the intent requirement."
You can be of the opinion that this shows bias, but it's a far cry from the Chinese censorship.
Clichés like "freedom is relative" are not serious arguments. You can't genuinely argue that the US, for all its problems, is less free than China. Words have meaning.
E.g. looking at freedom house index you can disagree on the precise score or how different aspects are weighed, but you can't argue with the underlying facts in the country reports
https://freedomhouse.org/country/scores
"Surveys of genocide scholars (e.g., one by the journal Journal of Genocide Research contributors) show meaningful disagreement, though a visible and growing share of specialists in genocide studies specifically have concluded the term applies or is defensible — more so than international law scholars generally, who tend to be more cautious about the intent requirement."
You can be of the opinion that this shows bias, but it's a far cry from the Chinese censorship.
Clichés like "freedom is relative" are not serious arguments. You can't genuinely argue that the US, for all its problems, is less free than China. Words have meaning.
E.g. looking at freedom house index you can disagree on the precise score or how different aspects are weighed, but you can't argue with the underlying facts in the country reports
https://freedomhouse.org/country/scores
You asked about genocide in general. Not Israeli genocide. Are you astroturfing? No one can make an omission that obvious
You can't genuinely argue that the US, for all its problems, is less free than China
This is how Americans convince themselves they are at the top of the world when they are not. No one outside of Europe and the US cares about the US anymore. They buy Chinese products and do business with Chinese businesses instead, right now. Worldwide, China is seen more positively than the US specially in global south countries. The exceptions of course are Europe and The US, the cold war first world countries of course.
freedom house index
This is obvious American bias. Astroturfing.
You can't genuinely argue that the US, for all its problems, is less free than China
This is how Americans convince themselves they are at the top of the world when they are not. No one outside of Europe and the US cares about the US anymore. They buy Chinese products and do business with Chinese businesses instead, right now. Worldwide, China is seen more positively than the US specially in global south countries. The exceptions of course are Europe and The US, the cold war first world countries of course.
freedom house index
This is obvious American bias. Astroturfing.
you live under a rock or what?
Try selling SaaS for finance (think Private Equity/Wall St type customers) that is powered by a Chinese model. See how far you get.
Hopefully they'll release some smaller models (<100B) that we can run on home hardware at faster than 10tok/s.
Its not as good as GLM 5.2 for agentic workflows while also being bigger. Competition is going to be ruthless because the super low cost to switching.
There is also AllenAi in the US, but they have yet to produce a model at this scale. Thankfully, new contenders can come out of nowhere and do well, as long as they can produce a competitive model.
There is also AllenAi in the US, but they have yet to produce a model at this scale. Thankfully, new contenders can come out of nowhere and do well, as long as they can produce a competitive model.
> Its not as good as GLM 5.2 for agentic workflows while also being bigger
GLM 5.2 underwent extensive post-training and iteration since its original release to reach its current state. This seems like an extremely strong model for a first release, with a lot of potential for improvement, just like DS4.
Sometimes I wish Meta had stuck with Llama 4 a bit longer to see how much further it could be pushed.
GLM 5.2 underwent extensive post-training and iteration since its original release to reach its current state. This seems like an extremely strong model for a first release, with a lot of potential for improvement, just like DS4.
Sometimes I wish Meta had stuck with Llama 4 a bit longer to see how much further it could be pushed.
Llama 4 wasn't deemed a success, and Meta pivoted away as its now former head of AI couldn't demonstrate, nor even showed interest in, business profit.
They overspent on llama 3 anyway so money ran dry, LeCun is good at running research, but budgets didn't stretch. Meta isn't investing in frontier big models anymore.
They overspent on llama 3 anyway so money ran dry, LeCun is good at running research, but budgets didn't stretch. Meta isn't investing in frontier big models anymore.
> Meta isn't investing in frontier big models anymore
Yes they are. Meta Muse is their attempt.
It's below frontier performance at the moment but they are spending on getting there.
Yes they are. Meta Muse is their attempt.
It's below frontier performance at the moment but they are spending on getting there.
Llama 4 was a bad architecture.
Meta Spark is moderately promising but of course closed source.
Meta Spark is moderately promising but of course closed source.
This is a great point
[deleted]
Do you think American companies will secretly distill frontier models to build open weight ones?
unlikely I think, they're likely doing this to garner some interest in their company but they seem pretty interested in revenue (judging by the companies they're working with)
isn't that what Reflection is trying to be?
What about Meta?
Also the fact that China is building solar power like crazy: that makes it fantastically more well spirited an endeavor to wish well.
It’s what Meta was supposed to do but Llama fell of the wagon.
There’s also Prism
There’s also Prism
Open models are Communist and steal from good hard-working American innovators! Are you are an un-American Commie is_stats?!?
/s ... but I genuinely think that sentiment is the reason we'll never have open models in America (except for the low-end stuff Meta deigns to share). If it's not something long-established (public roads, hospitals, police, fire, etc.) having the government do anything a corporation can do is a non-starter.
/s ... but I genuinely think that sentiment is the reason we'll never have open models in America (except for the low-end stuff Meta deigns to share). If it's not something long-established (public roads, hospitals, police, fire, etc.) having the government do anything a corporation can do is a non-starter.
i refuse to root for our enemies, but otherwise you are correct.
> Inkling is not the strongest overall model available today, open or closed. Instead, a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and availability on Tinker for fine-tuning.
Open base models that can be fine tuned on Tinker is a great business model IMO. You (i.e. an enterprise) can own your own model & have it perform frontier-or-better at your task at potentially much lower cost and Thinking Machines gets to be your essential infra/service provider in this world.
Also,
> Inkling-Small matches or exceeds its larger sibling on many benchmarks — the result of improvements we made to the pre-training data and recipe for the smaller model.
Very cool! Excited to see the next generations of Thinky models.
Open base models that can be fine tuned on Tinker is a great business model IMO. You (i.e. an enterprise) can own your own model & have it perform frontier-or-better at your task at potentially much lower cost and Thinking Machines gets to be your essential infra/service provider in this world.
Also,
> Inkling-Small matches or exceeds its larger sibling on many benchmarks — the result of improvements we made to the pre-training data and recipe for the smaller model.
Very cool! Excited to see the next generations of Thinky models.
> that can be fine tuned on Tinker
Good source to understand why this is valuable?
Good source to understand why this is valuable?
Frontier models need to do everything for everyone. It's expected (though not often done) that smaller models fine-tuned on specific tasks can approach frontier performance on a specific area. [0]
Post-training/fine-tuning is not trivial and having it as a service might make it more accessible.
[0] https://surgehq.ai/blog/training-on-complexconstraints
Post-training/fine-tuning is not trivial and having it as a service might make it more accessible.
[0] https://surgehq.ai/blog/training-on-complexconstraints
If you want an LLM to have knowledge about something, the knowledge has to either exist in its weights or be provided to it in its context. Because context is expensive and limited, and models tend to get dumber the more their context is filled, there is usually more that you'd need to put into context than can reasonably fit in it in order for the model to answer questions about your data. So your options are basically
1.) stuff it into context
2.) figure out a way to determine what to put into context based off of what is being asked of the model (RAG)
3.) change the weights of the model to have knowledge of your data baked into it (fine tuning)
1.) stuff it into context
2.) figure out a way to determine what to put into context based off of what is being asked of the model (RAG)
3.) change the weights of the model to have knowledge of your data baked into it (fine tuning)
Do you think 3 is better than 1 & 2 as context gets larger? I think for smaller data sets its mostly fine no. It's an interesting bet by TM. End state does seem some form of continual learning (model weights update like dreaming)
Here's a pelican: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
Real test here would be using tinker to fine tune a tinker model to generate pelicans
well it's good they didn't train on the test!
Gist is returning 403
How is this still a valid test if there is a good chance they will now specifically train on it?
I’m afraid you’re going to have to start randomizing your benchmarks somehow. I’m sure these models are trained on this problem by now.
If they did, they launched early! If they didn’t, their training data contains a bunch of poorly executed pelicans on bikes by other models.
It's not a benchmark, it's a meme
To be fair, the "they're trained on this benchmark" response is also a meme.
What strikes me the most is just how many different tasks are involved in modern model design. It used to be the case that you come up with a new loss function, slight architecture changes, etc., run your train and eval loop, and publish the artifacts.
Now, there’s so much work to do just to keep up. It’s the ultimate red queen race. All of the 500 steps involved, each of which is its own little optimization loop, is sort of awe inspiring.
But obviously this inverts the previous rules that small teams run faster than big teams. AI requires a big team. It’s only once the team pushes past the 1000s that organizational inertia seems to become an issue. Because until then, there’s way too many pieces for even a dozen super stars.
Now, there’s so much work to do just to keep up. It’s the ultimate red queen race. All of the 500 steps involved, each of which is its own little optimization loop, is sort of awe inspiring.
But obviously this inverts the previous rules that small teams run faster than big teams. AI requires a big team. It’s only once the team pushes past the 1000s that organizational inertia seems to become an issue. Because until then, there’s way too many pieces for even a dozen super stars.
It's probably not that many people necessary for creating a decent model. Soofi was small team: https://news.ycombinator.com/item?id=48870978
Very preliminary testing so far, but there is something here, far beyond what the benchmarks suggest. Only ever saw such outperformance of public evals vs my private ones with Anthropic models and while it is far to early to make any judgement at this stage, this model will take up a lot of mine time in the coming weeks by the look of things. Only ever viewed Moonshot AIs models as something I'd be able to live with open-weight-wise (Z.AIs output simply does not perform as well in my task set), but this has the potential to be the second. If Mistral came out with something like this, I suspect every Europhile (me included) would never stop talking about it.
Quick and still very early update, the model has (with web search disabled which was verified via the reasoning traces) accurately answered a number of questions focused on very niche details (engine specific maintenance in certain newtimers, very niche bag construction and material details) that I have only ever seen Gemini 3 and 3.1 Pro get correct. Neither Fable 5, nor GPT-5.6 Sol or any other model by any other lab has ever provided accurate information without web access for these specific questions for which an objectively correct answer absolutely exists and is general knowledge if one is versed in the specifics.
Being ahead of Fable 5 in any task, that is not included in public benchmarks and thus could be overfitted for, is impressive to say the least. Last time a model exceeded the expectations I had based on the release notes to such an extent was Haiku 4.5, which I still wish we got a solid replacement for.
Being ahead of Fable 5 in any task, that is not included in public benchmarks and thus could be overfitted for, is impressive to say the least. Last time a model exceeded the expectations I had based on the release notes to such an extent was Haiku 4.5, which I still wish we got a solid replacement for.
For a first model, and given it's open, I am gaining some faith in American Open research labs again...
I couldn't test it since it's not on openrouter or something, but even if it's only as good as GLM5.1 that's more than good enough first attempt, I think.
Perhaps a lot more labs will catch up to ballpark frontier esque level soon, I am all for more competition in any field.
I couldn't test it since it's not on openrouter or something, but even if it's only as good as GLM5.1 that's more than good enough first attempt, I think.
Perhaps a lot more labs will catch up to ballpark frontier esque level soon, I am all for more competition in any field.
NVIDIA is building Nemotron
I don't want to say this but Nemotron is not worth running on any sillicon, given Nvidia has been doing it for 3+ years, if Nvidia instead gave away GLM or KIMI API for free no one would use Nemotron the reason it's so wildly used is because Nvidia offers a Free API...
It's nice to see a strong long context open weights model that is multi-modal.
There are many applications that will benefit from the strength in audio here and until z.ai and co work in visual this could be very strong for general agentic applications, though I see there's a bit of weakness in the benches for areas that might make that less true.
Like all models need to slap it in your harness and do proper evals on the tasks you care about.
There are many applications that will benefit from the strength in audio here and until z.ai and co work in visual this could be very strong for general agentic applications, though I see there's a bit of weakness in the benches for areas that might make that less true.
Like all models need to slap it in your harness and do proper evals on the tasks you care about.
MiniMax M3 and DeepSeek v4-Pro are highly capable long context open weight multi-modal models. But long-context is a trap, because performance still falls dramatically after 150k-200k context.
> But long-context is a trap, because performance still falls dramatically after 150k-200k context.
I often see this repeated, and it is not true task to task. I work on this daily and we have several tasks where long context is advantageous and our evals against a whole battery of models with different windows show it as being so.
This is why having good evals for the tasks you're working on is so important.
I do grant it's a good rule of thumb.
I often see this repeated, and it is not true task to task. I work on this daily and we have several tasks where long context is advantageous and our evals against a whole battery of models with different windows show it as being so.
This is why having good evals for the tasks you're working on is so important.
I do grant it's a good rule of thumb.
> But long-context is a trap, because performance still falls dramatically after 150k-200k context.
I'm not sure exactly what causes the difference, but this heavily depends on the model. In my experience with Opus 4.8, I can go well over 500k and still get extremely good results. A drastically different example was GLM-5.1, which worked great until about 100k and then turned insane almost immediately. They did fix that with 5.2, though.
I'm not sure exactly what causes the difference, but this heavily depends on the model. In my experience with Opus 4.8, I can go well over 500k and still get extremely good results. A drastically different example was GLM-5.1, which worked great until about 100k and then turned insane almost immediately. They did fix that with 5.2, though.
5.1 going insane was probably also a inference quirk. Because it sometimes remained coherent the entire 200k context length.
Seems like this is particularly good at instruction following, but not as strong at coding as others. It's always great to get more diversity of open weight models though! I'll need to test this out to see what its "personality" is like.
seems pretty dang snappy and I like it's tone/personality so far.
> look at today's hackernews frontpage and generate me a daily briefing report (create an artifact) to read later for today's nerd news
https://chat.home.jake.town/artifacts/019f679d-99e5-7000-b02...
> look at today's hackernews frontpage and generate me a daily briefing report (create an artifact) to read later for today's nerd news
https://chat.home.jake.town/artifacts/019f679d-99e5-7000-b02...
This is the best voice/tone I've seen from any model so far. It's using filler words and phrases in places that normal people would put them, rather than sounding like a corporate customer support agent!
For the most part it’s better than Nemotron, worse than GLM. This makes it the best American open weights model from what I can tell?
It's nearly double the size of Nemotron 3 Ultra, so I'd expect it to be considerably better, although the active parameter count seems to be a touch lower at 41B vs 55B
I'm surprised that Nemotron gets mentioned at all. In my experiments with it for coding tasks it performed extremely poorly, essentially unusable.
I focus on realtime voice AI uses cases and nemotron's time to first token is INSANELY fast. It's become a legit option for voice use cases
it is pretty good at instruction following and has extremely fast decode.
What are you basing those comparisons on?
“Alongside Inkling we are sharing a preview of Inkling-Small, a 276B-parameter Mixture-of-Experts model (12B active, vs. 41B for Inkling) with a different performance/latency trade-off.”
Buried at the end there is the details I was most interested in - a possible competitor for DeepSeek V4 Flash? Excitedly awaiting the release of the weights for this one.
Buried at the end there is the details I was most interested in - a possible competitor for DeepSeek V4 Flash? Excitedly awaiting the release of the weights for this one.
I never thought i'd see the day they released a model, rather than a blog post. The Figure 3 demo being a screencap of chrome in localhost made me feel better about myself. Jokes aside, best western open weights model- very cool.
They are one of the few labs (perhaps even the only one at this level) that are doing something both unique and useful, rather than simply imitating what the others are doing: https://thinkingmachines.ai/blog/interaction-models/
Interestingly, when opening this page, the first thought I had was not that the benchmarks should be high, but 'I really hope they did not benchmaxx'. I think a model with modest benchmark scores can have much better real world utility as opposed to the current frontiers that are RL'd into being robotic and rigid.
What are the different business models for open-weight AI companies?
For thinking machines, they provide super simple finetuning APIs.
if it is their model, they can have more lower level integrations for that. Thinking machines might be the only large lab in the US to have business interest aligned with open sourcing strong models that are customizable.
if it is their model, they can have more lower level integrations for that. Thinking machines might be the only large lab in the US to have business interest aligned with open sourcing strong models that are customizable.
Just serving the model over API seems like a natural fit and is what many of them are doing. So simply being the cloud provider for your own open weight model can be a source of revenue
But so can everyone else. What’s the moat for spending all those billions. I understand the Chinese angle, they need to undermine American models as a matter of statecraft, but what is the business model here? It just seems like VC charity.
use open models to gain marketing/users/attention and then go closed? maybe
There are no moats. LLM's are a commodity. The point in spending all of the billions is to have strong domestic open-weight models.
One of the worst case scenarios regarding LLM's is monopoly control, so these billionaires know they need to invest in competition.
One of the worst case scenarios regarding LLM's is monopoly control, so these billionaires know they need to invest in competition.
3848488459(1)
What is the moat? The time it takes for AI to rewrite an efficient inference stack for a new model? Considering most LLMs follow a similar architecture, adapting to a new model shouldn't take that much time.
There is no moat. At the moment, all of these companies are burning money to gain mindshare and market share. That's what Thinking Machines is doing; they're not looking for a business model.
There might be moats. More-efficient training or inference, for instance.
Nobody in the LLM world has a moat, or even an actual business model
I don't know why people keep saying there's no moat. There's no moat. Having a FUCK ton of money to train these gigantic fucking models and retain the brains to make it happen is a moat.
You're not going to train one using a VPS from LowEndBox.
You're not going to train one using a VPS from LowEndBox.
But if people can download it for free, that is a drawbridge across the moat competitors can use to get the same model.
Maybe the thesis is that
Open source low cost models will dominate most enterprise tasks as cost curves will dictate usage. TM is trying to replicate that especially as the US and China gets more defensive with their tech
Open source low cost models will dominate most enterprise tasks as cost curves will dictate usage. TM is trying to replicate that especially as the US and China gets more defensive with their tech
Similar to companies working on FOSS codebases, hosting (sometimes with the license restricting third-parties in some way), providing tailored models and services to customer's and getting bought for your team if your model happens to be competitive enough.
- inference
- RLaaS (Tinker, or the more involved FDE motion a la Reflection / Applied Compute)
- RLaaS (Tinker, or the more involved FDE motion a la Reflection / Applied Compute)
The actual part on fine-tuning seems very short in the article. Did I miss a page where they have examples of fine-tuning it for different niche use cases?
Optimizing models to be fine-tuned is an amazing direction, but just makes me wonder how much better this actually is at being fine-tuned compared to other models. As none of the modern models are great at being fine-tuned afaik. Basically looking for some sort of benchmark showing that it's resistant to overfitting / catastrophic forgetting, etc.
Would be very interesting to see concrete demonstrations of different fine-tunes of the model. I'd imagine they've done hundreds of those internally.
Optimizing models to be fine-tuned is an amazing direction, but just makes me wonder how much better this actually is at being fine-tuned compared to other models. As none of the modern models are great at being fine-tuned afaik. Basically looking for some sort of benchmark showing that it's resistant to overfitting / catastrophic forgetting, etc.
Would be very interesting to see concrete demonstrations of different fine-tunes of the model. I'd imagine they've done hundreds of those internally.
They have good docs on finetuning in general here: https://thinkingmachines.ai/tinker/
I used it last week for an application using a small model just as an experiment. It all went very well. The model did not turn out to be good though because my training data was of bad quality. I plan to work on it more this weekend.
I used it last week for an application using a small model just as an experiment. It all went very well. The model did not turn out to be good though because my training data was of bad quality. I plan to work on it more this weekend.
I think Inkling actually makes sense as a sustainable open-weight model!
A few months ago, I tried using their Tinker product to finetune/post-train an open weight model, but I found it easier and cheaper to just build from scratch.
Ideally, I want to test RL environments on a small model and then scale up to a large model. That's hard to do right now, in part because there's no good family of open-weight models with a great small and large MoE variant [1]. It's extremely time consuming and hard to build long-context training infra for different models with different architectures.
If they go all-in on integrating Tinker with their own models and building efficient long-context training I could see myself using Tinker with Inkling.
Because of that, I think building their own models actually makes sense! And it seems like there's a corresponding business model to support it.
[1] Qwen models are the closest: Qwen 3.6 35B A3B is a capable small MoE model, and larger open weight Qwen models exist. But they aren't as performant as the alternatives, because their frontier models – like Qwen 3.6 Plus – are no longer open weight.
A few months ago, I tried using their Tinker product to finetune/post-train an open weight model, but I found it easier and cheaper to just build from scratch.
Ideally, I want to test RL environments on a small model and then scale up to a large model. That's hard to do right now, in part because there's no good family of open-weight models with a great small and large MoE variant [1]. It's extremely time consuming and hard to build long-context training infra for different models with different architectures.
If they go all-in on integrating Tinker with their own models and building efficient long-context training I could see myself using Tinker with Inkling.
Because of that, I think building their own models actually makes sense! And it seems like there's a corresponding business model to support it.
[1] Qwen models are the closest: Qwen 3.6 35B A3B is a capable small MoE model, and larger open weight Qwen models exist. But they aren't as performant as the alternatives, because their frontier models – like Qwen 3.6 Plus – are no longer open weight.
I think this actually makes sense as a sustainable product for Thinking Machines!
A few months ago, I tried using Tinker to finetune/post-train an open weight model, but I found it easier and cheaper to just build from scratch.
Ideally, I want to debug RL environments on a small model and then scale up to a large model. That's hard to do right now, in part because there's no good family of open-weight models with a great small and large MoE variant [1].
Concretely, my go-to small model for finetuning is Qwen 3.6 35B A3B. To test with a large model, I need to switch to Kimi K2.6 or GLM 5.2. It's extremely time consuming and hard to build long-context training infra for different models with different architectures.
In theory, their Tinker product already solves that with a consistent training API over several open weight models. But when I tried to use it, I found it easier and cheaper to just build from scratch. If they go all-in on integrating Tinker with their own models and building efficient long-context training, I could see myself using Tinker with Inkling.
Because of that, I think building their own models actually makes sense! And it seems like there's a corresponding business model to support it.
[1] Qwen models are the closest: Qwen 3.6 35B A3B is a capable small MoE model, and larger open weight Qwen models exist. But they aren't as performant as the alternatives, because their frontier models – like Qwen 3.6 Plus – are no longer open weight.
A few months ago, I tried using Tinker to finetune/post-train an open weight model, but I found it easier and cheaper to just build from scratch.
Ideally, I want to debug RL environments on a small model and then scale up to a large model. That's hard to do right now, in part because there's no good family of open-weight models with a great small and large MoE variant [1].
Concretely, my go-to small model for finetuning is Qwen 3.6 35B A3B. To test with a large model, I need to switch to Kimi K2.6 or GLM 5.2. It's extremely time consuming and hard to build long-context training infra for different models with different architectures.
In theory, their Tinker product already solves that with a consistent training API over several open weight models. But when I tried to use it, I found it easier and cheaper to just build from scratch. If they go all-in on integrating Tinker with their own models and building efficient long-context training, I could see myself using Tinker with Inkling.
Because of that, I think building their own models actually makes sense! And it seems like there's a corresponding business model to support it.
[1] Qwen models are the closest: Qwen 3.6 35B A3B is a capable small MoE model, and larger open weight Qwen models exist. But they aren't as performant as the alternatives, because their frontier models – like Qwen 3.6 Plus – are no longer open weight.
Do you think the barbarians are at the gates of OpenAI and Anthropic? If cheaper, open weights models can seriously take revenue away from those two labs for (frontier - 1) model use cases (which are the models most enterprises will choose) then OpenAI and Anthropic are left only with users using their latest and greatest model AND who will keep upgrading to the newer ones?
Bull case: iteration becomes so quick that frontier-1 won't cut it. OpenAI and Anthropic are both betting on the singularity, I suppose.
The "singularity" as stated requires AI to either make a technological advancement strong enough to be deadly to humans (besides just intelligence), or spreading deep institutional support for itself among society.
The idea that "we will get superintelligence first, then... ???" is kind of a weird notion. I mean, it's pretty arguable that we do have at least some form of superintelligence. The AI itself needs to actually do something with it though. Either that, or more likely, someone needs to do something bad with the superintelligence.
That could be both re-assuring or not. Because under that view, given how AI is being integrated so quickly into society, it's not going to take this fantasy view of superintelligence to reach the singularity. If you have broad institutional support and crowd out the thing we call humanity over time (the two ways to 'solve' a problem: solve it, or declare it meaningless), that is another way to reach the singularity.
The idea that "we will get superintelligence first, then... ???" is kind of a weird notion. I mean, it's pretty arguable that we do have at least some form of superintelligence. The AI itself needs to actually do something with it though. Either that, or more likely, someone needs to do something bad with the superintelligence.
That could be both re-assuring or not. Because under that view, given how AI is being integrated so quickly into society, it's not going to take this fantasy view of superintelligence to reach the singularity. If you have broad institutional support and crowd out the thing we call humanity over time (the two ways to 'solve' a problem: solve it, or declare it meaningless), that is another way to reach the singularity.
Without the singularity, I think Frontier labs will offer intelligent model blends. They'll have their own versions of "cheap" models and be expert at using the appropriate amount of compute for a task.
competition in this space is great, especially with open models/weights. I think the answer is not closed source models. Similar to the Unix versus Linux situation in the 1990's, open source wins out. Yesterdays story about how OpenAI has now began encrypting traffic between model and agent [0], this story brings a breath of fresh air. There is nothing "Open" about hiding the communication between model and agent, especially with software that is running within a trusted environment/network. It needs to be more transparent, not less.
[0] https://www.theregister.com/ai-and-ml/2026/07/15/openai-hide...
[0] https://www.theregister.com/ai-and-ml/2026/07/15/openai-hide...
Open Source won out because the cost of compute fell through the floor. I'm not sure whether we're going to see a similar dynamic play out this time, although I would greatly prefer it to.
cost of compute, also the less likelyhood of underhanded tactics or vendor lock-in
Smart that she says "not the strongest overall model, open or closed". This is a rare for an AI lab to say out loud. They basically decided to compete on customizability, and not on topping the temporary leaderboard. Also corroborates what we recently wrote: any Lab's capability lead cant hold for long anyway, it's a "red queen race" that never settles: https://news.ycombinator.com/item?id=48892559
Give me a good 180B param model that fits snuggly on an single DGX spark and I will sing your praises.
the gap between 'open weights' and 'open source' is now wide enough to fit an entire corporate legal department
This is a winner IMO. Lots of cost pressure on token spend atm within enterprises and tasks that don't require Opus / Codex class models.
These companies have hopefully captured all of their traces and now have enough to fine-tune an open model and host themselves.
Inkling feels like the right base - not obsessed with benchmaxxing on coding but rather being adaptable to the task required
For tasks like GTM, support, content writing etc. seeing 80%+ savings
These companies have hopefully captured all of their traces and now have enough to fine-tune an open model and host themselves.
Inkling feels like the right base - not obsessed with benchmaxxing on coding but rather being adaptable to the task required
For tasks like GTM, support, content writing etc. seeing 80%+ savings
Open weight models as a category might be a winner due to cost pressure, but I don't think Inkling is a top performer in that respect. Pricing on a token basis is 6-9x higher depending on the provider.
Can chime in on the support use case specifically: GPT OSS performs really well here and has been somewhat of a benchmark with our customers, limited testing [0] against Inkling reveals basically identical performance, but with a significant cost increase at scale.
I'd say that for real-world tasks that aren't coding most companies don't see value by being on the latest and greatest model.
[0] https://valiopt.com/blog/inkling-model-customer-support-revi...
Can chime in on the support use case specifically: GPT OSS performs really well here and has been somewhat of a benchmark with our customers, limited testing [0] against Inkling reveals basically identical performance, but with a significant cost increase at scale.
I'd say that for real-world tasks that aren't coding most companies don't see value by being on the latest and greatest model.
[0] https://valiopt.com/blog/inkling-model-customer-support-revi...
Looks like it can be tried at https://tinker.thinkingmachines.ai/playground
How much mortgage equity would I need to do that 27min fine tune demo on local :)
Self fine tuning like that though seems like a whole new set of possibilities unlocked.
Self fine tuning like that though seems like a whole new set of possibilities unlocked.
Is this the same model they are using for interaction models?
https://thinkingmachines.ai/blog/interaction-models/
https://thinkingmachines.ai/blog/interaction-models/
yes. At least they want to use it for that, I am not sure they already do.
(look under "multimodality" in the blog post: https://thinkingmachines.ai/news/introducing-inkling/)
(look under "multimodality" in the blog post: https://thinkingmachines.ai/news/introducing-inkling/)
is there something in the space of "taskifying" enterprises data for them? inkling on its own looks high-quality, but expecting companies to spend $ figuring it out before spending more $ on the actual fine-tuning job seems ... hard, especially if making the model especially customizable is the goal? or do the unit economics just work out with a small number of fine-tuners training a ~1T model on Tinker?
It looks like HuggingFace shows Apache-2.0 but they have AUP. How does it work together?
Giving the accuracy-token graph and not the accuracy-cost graph, thus we cannot easily compare costs with other models, is not a way to gain my trust
This seems like a really really great debut model for a new lab. I'm happy
Happy to see an open weight model ! This has all the right ingredients for success.
The Artifical Analysis has a link on their homepage but it 404's :/
https://artificialanalysis.ai/models/inkling
https://artificialanalysis.ai/models/inkling
They also indicate they have a 276B A12B version, but it doesn't seem the weights are available. This might actually be able to fit in 128GB when quantized to 2 bits or so which makes it interesting.
They mention in the announcement link https://thinkingmachines.ai/news/introducing-inkling/ that they are still testing Inkling-Small and it will also still be multimodal. This makes it super interesting as a Deepseek V4 Flash replacement (and would be interesting with DwarfStar / ds4 if it gets supported).
Interested in the implied strategy - that training a bespoke model for what you need will make economic sense over using a mass-trained model. I wonder if that's true?
Same. Gutsy bet to make in the face of Fable / Mythos, but the multimodal quality is at least a promising technical/ product story to tell. Everyone knows throwing Opus at everything is wasteful and domain expertise should live in the weights eventually; the question is whether foundation model scaling will slow down enough soon enough for that to matter.
Or maybe this is just a warm-up / stopgap and Thinking Machines is betting on finding the next architectural breakthrough that lets it compete with the big foundation models?
Or maybe this is just a warm-up / stopgap and Thinking Machines is betting on finding the next architectural breakthrough that lets it compete with the big foundation models?
My personal bet is that this model should really shine in Autoresearch NanoGPT-style speedruns because its first-class integration with Tinker
open-weights but not open-source. we're one step away from 'open-vibes' as a license category
Do they have an api to try the model in real envs?
They've got an openai + anthropic compatible endpoints. I got far enough to run some tests on the openai endpoint, albeit with some finagling (their /models list is empty, my tool auto-configures using that, was an initial stumble).
Thanks! I found the OpenAI-compatible endpoint and got it working. I ran Inkling on a couple of my own evals. It looks promising, but on my cases it still fell short of GPT-5.4 and GPT-5.6 Luna.
I tried Hy3 today and liked it. It's a (small) step up from DSV4P.
Something on that level but multi-modal would be quite nice!
Something on that level but multi-modal would be quite nice!
Lol slither.io is the new benchmark now? I guess my game slitherworld.com is now something that can be vibecoded too
too bad we'll never know how good it is, since they used a radar plot to show its benchmark scores!
How does the radar plot prevent you from looking at just one of its axes?
Very impressive model, exciting to see an American open-source lab with such competitive results.
If it's ~30% bigger and not as good as GLM 5.2, why would I tinker with this model?
Maybe for the multi modal?
Maybe for the multi modal?
> If it's ~30% bigger and not as good as GLM 5.2, why would I tinker with this model?
The benchmarks never tell the full story. Some of the open weights models have been benchmaxxed for a while. Their utility on real work can be different than the benchmark number.
The multimodal input is also a big deal. Having vision input is really helpful for a lot of tasks.
The benchmarks never tell the full story. Some of the open weights models have been benchmaxxed for a while. Their utility on real work can be different than the benchmark number.
The multimodal input is also a big deal. Having vision input is really helpful for a lot of tasks.
I second that. Gemini 3.5 Flash rocks the benchmark charts but is terrible as an agent. Horrible instruction adherence and makes WAY too many tool calls
Gemini 3.1 pro is still better at creative writing than any other model.
Then why are they publishing the benchmarks which makes them look worse than GLM 5.2?
Because it's still informative
being close is still impressive, especially for their first (released) model
gives me hope that the training moat is even smaller than we thought
gives me hope that the training moat is even smaller than we thought
I'm not sure why I'm being downvoted but I didn't mean it in a negative way.
For such announcement, I would expect them to give me clues on when I should use this model and in which cases it's the best one.
The benchmarks that they share doesn't indicate that it's cheaper to run than other models, or can fit in my local machine, or excels in a specific vertical.
After reading the comments here and X, I can see it being the top-3 multi-modal open-source model though.
For such announcement, I would expect them to give me clues on when I should use this model and in which cases it's the best one.
The benchmarks that they share doesn't indicate that it's cheaper to run than other models, or can fit in my local machine, or excels in a specific vertical.
After reading the comments here and X, I can see it being the top-3 multi-modal open-source model though.
If they have a really seamless fine-tuning experience and maybe can help you extract the data you need to FT (which is one of the big challenges in actually getting fine-tuning democratized), maybe you would use it because "Tinker" defaults to it.
The model could also be more flexible for non-coding use-cases (they show the results for reasoning being strong) so maybe the argument is to use it for non-coding use-cases to drive relatively deterministic conclusions for non-coding agents (they have also done some determinism work on kernels, which could be useful in pulling on that thread of deterministic models that are fine-tuned for everything that is not writing code.)
That said, I'm not sure how much all the work they have done actually synergizes or if the market size (at least in the short to medium term) is big enough for a huge outcome from the company's current valuation with those bets as the enterprise agent estate is taking a while to evolve. Hence companies like Anthropic and OpenAI are throwing tons of consulting money at the problem.
The model could also be more flexible for non-coding use-cases (they show the results for reasoning being strong) so maybe the argument is to use it for non-coding use-cases to drive relatively deterministic conclusions for non-coding agents (they have also done some determinism work on kernels, which could be useful in pulling on that thread of deterministic models that are fine-tuned for everything that is not writing code.)
That said, I'm not sure how much all the work they have done actually synergizes or if the market size (at least in the short to medium term) is big enough for a huge outcome from the company's current valuation with those bets as the enterprise agent estate is taking a while to evolve. Hence companies like Anthropic and OpenAI are throwing tons of consulting money at the problem.
There's also an Inkling-Small that is 276B, 12B active that is much smaller than GLM 5.2 and still multimodal. Not released yet, but in the announcement link they mention that they're testing Inkling-Small & will release as open weight after testing. That one may be interesting as a Deepseek V4 Flash replacement.
> Maybe for the multi modal?
Yeah
Yeah
"Our model has TRILLIONS of parameters!"
"But it's worse than Mistral 7b"
cape
"But it's worse than Mistral 7b"
cape
The most important observation is that open source matches their business model, which is to provide fine tuning services for enterprises, etc.
That what makes this a (potentially) safer model to build on top of
That what makes this a (potentially) safer model to build on top of
Your first mission should be providing a working dark mode site.
Holy flashbang.
Holy flashbang.
Excited to try out its capability, especially audio and video.
It's nice that it has a long context window, but in practice, I find I always have to clear context btw 150k-400k context even if the context window is 1M on paper.
It's nice that it has a long context window, but in practice, I find I always have to clear context btw 150k-400k context even if the context window is 1M on paper.
45 trillion tokens is a lot
missed oppotunity to start a vacuum cleaner company imo
the open-weight model release cadence is approaching npm package velocity. soon we'll have left-pad-7b and someone will unpublish it and break half of production
audio??? can it listen to music
Splatoon LLM
[deleted]
The MoE design largely follows DeepSeek-V3
artificial analysis puts it quite low below previous gen open weights models from China. Why are people so ecstatic?
everyone and their grandma shipping top tier models now. anthropic and openai trying to capture the app layer with their shitty 'super app'
We always have been, Big Tech has been extremely slow to catch up to the indies.
Nobody is making money lmao.
I would not bet on OpenAI creating any good products, they never have. They are like Meta in all this - never innovated anything themselves, can only acquire others to stay relevant. They'll never do an incredible consumer experience on the level of a PlayStation or Blizzard or even Google.
Nobody is making money lmao.
I would not bet on OpenAI creating any good products, they never have. They are like Meta in all this - never innovated anything themselves, can only acquire others to stay relevant. They'll never do an incredible consumer experience on the level of a PlayStation or Blizzard or even Google.
Cool, now we just need the GPU that supports it
I think we’re going to start seeing more OSS models that perform especially well on certain tasks instead of trying to be generalists like the frontier models. That’s a winning formula because if you’re building an app on a model it often has a specific set of use cases
I really respect the epistemtics work here. It might become an accurate, inexpensive open-weight workhorse for high-level prioritization and decision-making work. (Finance bros will also love this)
You certainly cooking smth, Good Luck Mira.
[flagged]
We detached this comment from https://news.ycombinator.com/item?id=48929037 and marked it off topic.
You may be new here. This is Simon’s de facto benchmark for models. I happen to find it a really good one.
Small aside: It’s crazy to me that while it’s improved over time it does seem like most of the models haven’t been trained specifically to defeat this one.
Small aside: It’s crazy to me that while it’s improved over time it does seem like most of the models haven’t been trained specifically to defeat this one.
The pelican thing is definitely getting deeper into model weights over time just by getting fed threads like this.
Welcome to HN :)
why is this website ai slop
Is it really that bad? I always get the impression that their blog posts look especially beautiful with their font choices and overall design. They are typographically pleasing, and if I could, I would use this as the distraction-free reading mode for every web page.
It feels like I’m reading a newspaper, but oddly, without them resorting to any skeuomorphic tricks.
It feels like I’m reading a newspaper, but oddly, without them resorting to any skeuomorphic tricks.
do you want them to focus on the website or their model?
do you buy a device because of its unboxing experience?
Raised 2 billion dollars at a 12 billion valuation and debuts at 41 on the Artificial Analysis Intelligence Index, while KIMI and DeepSeek will release Fable-class models this week. What a joke.
Moonshot (Kimi) has raised $3.77B and been around for >3 years, Thinking Machines raising $2B and releasing a decent open weights model in 16 months is actually quite comparable.
I think your comments might be overly negative. Would you expect the first model from an organization to top the chart?
It's a process, I think they did pretty good. They have enough resources to continue improving on it
> ...while KIMI and DeepSeek will release Fable-class models this week.
What new model is DeepSeek releasing? Their current V4 Pro at Max reasoning is consistently worse than GLM 5.2 at Max reasoning, though the latter is close to Opus 4.8 at Extra/Max reasoning, albeit a little bit worse in my experience (though if they gave comparable amounts of tokens to Anthropic 5x Max subscription I could see myself moving over, currently they give you less though even with their ZCode discount).
In practical agentic development, none of those seem to be that close to Fable to me. Spent 181 million tokens with GLM 5.2 with ZCode in the past month, 142 million with DeepSeek V4 Pro with ZCode and OpenCode and about 3.45 billion across all Anthropic models with Claude Code, though understandably with my workload between 95-99% of them are cached (very docs/plan/tooling/read heavy work to limit slop, albeit with sub-agents and workflows).
What new model is DeepSeek releasing? Their current V4 Pro at Max reasoning is consistently worse than GLM 5.2 at Max reasoning, though the latter is close to Opus 4.8 at Extra/Max reasoning, albeit a little bit worse in my experience (though if they gave comparable amounts of tokens to Anthropic 5x Max subscription I could see myself moving over, currently they give you less though even with their ZCode discount).
In practical agentic development, none of those seem to be that close to Fable to me. Spent 181 million tokens with GLM 5.2 with ZCode in the past month, 142 million with DeepSeek V4 Pro with ZCode and OpenCode and about 3.45 billion across all Anthropic models with Claude Code, though understandably with my workload between 95-99% of them are cached (very docs/plan/tooling/read heavy work to limit slop, albeit with sub-agents and workflows).
DeepSeekV4 was a preview model, read the papers. It's not the final model. They released it to demonstrate architectural capabilities. They are still training and the model release is planned within the next month.
If they somehow make it approach Fable in capability, I’ll be quite surprised!
I haven't used Fable, but if the hype is to be believed then it's a jump in model capability. If so then I don't expect the next DeepSeekV4 version to match it. However, if the next DSV4 version get's the kind of jump 3.1 got over 3.0 or 4 got over 3.2, I'll be very happy with it. Progress is progress. We "can" run DSV4 locally, Fable is closed.
Not even GPT 5.6 approaches it, why would DeepSeek with less investment?
Well Chinnese companies are way more efficient, Deepseek did manage to get quite far with a relatively tiny number of staff and limited funding. Then you have companies like Meta which suck pretty much at everything they (excluding their core business) regardless of how much money they throw at it.
I prefer 5.6 Sol to Fable personally and I've used both extensively.
Anyone would think these investors are making a bet they can improve using that cash.
How does this compare with Grok 4.5, Fable, GPT 5.6 etc I can use them for a few bucks a month, whats the benefit of using this model? is it more intelligent, faster, cheaper (and no I don't want to spin-up my own mini datacenter to run it 'in house') I want to install a harness/app/visit web page auth and start as 99% of people using AI want to do.
If you want to run locally, checkout https://github.com/danielhanchen/llama.cpp/tree/add-inkling https://unsloth.ai/docs/models/inkling https://huggingface.co/unsloth/inkling-GGUF https://huggingface.co/unsloth/inkling-NVFP4
This supposedly is better than KimiK2.7, as much hype as GLM5.2 gets, I find myself using KimiK2.7 half of the time, so if the benchmark is true, then this can definitely go in the mix. My hope is that it might have strengths in some areas to beat all other open weight models.