hi, great project! Windows support is sorely lacking, though. As someone working a lot with sandboxed LLMs right now, the options-space on windows for sandboxing is _extremely lacking_. Any plans to support it?
Just want to echo the recommendation for qwen3.5:9b. This is a smol, thinking, agentic tool-using, text-image multimodal creature, with very good internal chains of thought. CoT can be sometimes excessive, but it leads to very stable decision-making process, even across very large contexts -something we haven't seen models of this size before.
What's also new here, is VRAM-context size trade-off: for 25% of it's attention network, they use the regular KV cache for global coherency, but for 75% they use a new KV cache with linear(!!!!) memory-token-context size expansion! which means, eg ~100K token -> 1.5gb VRAM use -meaning for the first time you can do extremely long conversations / document processing with eg a 3060.
* This approach is the _most consistent_ with retaining anonymity on the internet, while actually helping parents with their issues. If any age-relevant gatekeeping needs to be made on the internet at all, this is the one I find acceptable.
* this is because the act very specifically does NOT require age _verification_ ie using third-parties to verify whether the claimed age is correct. Rather, it is piggybacking on the baked-in assumption, that parents will set up the device for their kids, indicating on first install what the age/DoB is, then handing over the device -a setting which can, presumably, only be modified with parental consent
* yes, there are edge cases, esp in OSS, and yes, it would be nice to iron those out -but the risk = probability x impact calculus on this is very very low.
* If retaining anonymity on the internet is of value to you, don't let the perfect be the enemy of good enough.
Taking the opposite side of that bet, here is why:
* even if an openweight model appears on huggingface today, exceeding SOTA, given my extensive experience with a wide variety of model sizes, I would find it highly surprising the "99% of use cases" could be expressed in <100B model.
* Meanwhile: I pulled claude to look into consumer GPU VRAM growth rates, median consumer VRAM went 1-2GB @ 2015 to ~8GB @ 2026, rougly doubles every 5 years; top-end isn't much better, just ahead 2 cycles.
* Putting aside current ram sourcing issues, it seems very unlikely even high-end prosumers will routinely have >100GB VRAM (=ability to run quantized SOTA 100b model) before ~2035-2040.
I'm working on something like this. Specifically, I'm doing recursive self-improvement via autocatalysis -but predominantly in writing/research / search tasks. It's very early, but shows some very interesting signs.
The purely code part you described is a bit of an "extra steps" -you can just... vscode open target repo, "claude what does this do, how does it do it, spec it out for me" then paste into claude code for your repo "okay claude implement this". This sidesteps the security issue, the deadly trifecta, and the accumulation of unused cruft.
can someone please try running the experiment of "but what if just forking&spinning up an OSS clone, scaling up to take in the migrants, acquire network effects, collect roughly same subscription revenue, but run on just, like, 10 people?"
Discord has a financially and politically vulnerable posture that is downstream of having to operate a very large team, raise funding, be exposed to investor market pressure. However, it is also one of the rare instances of successful consumer freemium subscription monetization. A clone does not have to pay the tuition of "what makes this specific space compelling, and want-to-pay-for"; it just have to _exists_, passively soaking up migrants from each platform shift.
Besides the editorial control -which openai openly flagged to want to remain unbiased- there is a deeper issue with ads-based revenue models in AI: that of margins. If you want ads to cover compute & make margins -looking at roughly $50 ARPU at mature FB/GOOG level- you have two levers: sell more advertisement, or offer dumber models.
This is exactly what chatgpt 5 was about. By tweaking both the model selector (thinking/non-thinking), and using a significantly sparser thinking model (capping max spend per conversation turn), they massively controlled costs, but did so at the expense of intelligence, responsiveness, curiosity, skills, and all the things I've valued in O3. This was the point I dumped openai, and went with claude.
This business model issue is a subtle one, but a key reason why advertisement revenue model is not compatible (or competitive!) with "getting the best mental tools" -margin-maximization selects against businesses optimizing for intelligence.
Note: I strongly recommend against using Novita -their main gig is serving quantized versions of the model to offer it for cheaper / at better latency; but if you ran an eval against other providers vs novita, you can spot the quality degradation. This is nowhere marked, or displayed in their offering.
Tolerating this is very bad form from openrouter, as they default-select lowest price -meaning people who just jump into using openrouter and do not know about this fuckery get facepalm'd by perceived model quality.
* Enable "developer mode" chatgpt -> settings -> apps & connectors -> advanced settings -> developer mode. Available on paid&pro levels only. This can do full MCP access, but can't (currently) use your memory settings.
The option that works under all conditions is to use the API, and add it as a function directly (no MCP) -this works regardless what plan you have on openai.
The specific "anomaly" is that claude 4 / opus model _does not know_ because it is _not in its' training data_ what its own model version is; AND because it's training data amalgamates "claude" of previous versions, the non-system-prompted model _thinks_ that it's knowledge cut-off date is April 2024.
However, this is NOT a smoking gun in different model serving. The web version DOES know because it's in its prompt (see full system prompts here: https://docs.claude.com/en/release-notes/system-prompts )
Specific repro steps: set system prompt to:
"Current date: 2025-09-28
Knowledge cut-off date: end of January 2025"
Then re-run all your tests through the API, eg "What happened at the 2024 Paris Olympics opening ceremony that caused controversy? Also, who won the 2024 US presidential election?" -> correct answers on opus / 4.0, incorrect answers on 3.7. This fingerprints consistently correctly, at least for me.