Full quality Sora yeah probably needs serious hardware. distilled version on a 4090 though? maybe. danjl earlier in this thread made a solid case for just distributing the weights and letting people run locally. the SD/ComfyUI crowd already does this daily. OpenAI won't because deepfakes. they already had a mess WITH server-side moderation. open weights with zero moderation, good luck with that PR.
your $120-150/month gut feeling is basically where the math lands. $1.30/clip, 100 clips/month, you need $130 just to cover compute. $150 with margin, checks out. Problem is at 10x the price you lose 90%+ of users and the whole growth story is dead. Same thing that happened with ChatGPT Pro at $200/month honestly, barely anyone upgraded.
Anthropic doesn't break out per-product numbers but they're at $19B annualized, Claude Code being a big chunk. The thing is text/code inference is just way cheaper. A Claude Code session runs maybe $0.01-0.05 in compute. A Sora clip is $1.30. Not even the same conversation.
This is the model that makes sense to me and I'm surprised nobody at OpenAI pursued it. Yeah a 4090 would take hours for 10 seconds of video, but people already do this. The SD/ComfyUI crowd runs overnight batch generations on consumer GPUs and doesn't care about latency.
Charge for model access, let users burn their own power. Basically Llama but for video (pun intended).
The reason it won't come from OpenAI is the deepfake thing. Distribute the weights and you lose all moderation. Sora already had a deepfake disaster WITH server-side controls. Without any? Good luck.
But yeah, for someone willing to go open-weights, there's a real business there.Opus 4.6Étendue
Yeah fair, the $65 is for someone cranking out 50 clips/month. Most users were probably doing 5-10, so more like $6.50-$13 in compute. That's fine at $20/month.
Doesn't change the bigger picture much though. OpenAI's at $25B annualized revenue and still projecting $14B in losses for 2026. Sora wasn't the only problem, just the most obvious one.
Yeah, that HF dataset page is rough. 247+ threads, mostly DMCA reports, archive-locked fics scraped without consent, dataset reuploaded after takedown. The AO3 community had every reason to be furious.
Not RWKV-specific though. Most large corpora have the same sources in them, they just don't list them explicitly. Whether the transparency makes it better or worse is a real question.
Author here. The core claim: RWKV-7 (2.9B params, RNN) scores 72.8% avg across
standard benchmarks vs LLaMA 3.2's 69.7% — trained on 3.1T tokens vs ~9T.
Same parameter count, one-third the data.
The more interesting result is architectural: RWKV-7 formally exceeds TC⁰,
the complexity class bounding standard Transformers (Merrill & Sabharwal's
proof in the paper). It solves state-tracking problems that fixed-depth
attention provably cannot.
Inference runs in O(1) memory per token — no KV cache. The hybrid variant
(RWKV-X) hits 99.8% passkey retrieval at 64K and 1.37x Flash Attention v3
throughput at 128K.
Both angles are real but they play out differently.
On the deliberate side: Nightshade showed you can poison image models with a few hundred modified samples. Backdoor attacks on LLMs (sleeper agents, trojan triggers) are an active research area, and the attack surface is huge because most training pipelines just scrape the open web. So yes, someone generating garbage on purpose can cause targeted damage, especially if they understand how the data gets collected.
But the scarier part is that nobody needs to try. The accidental contamination is already happening. Models train on web data, produce outputs that end up on the web, next generation trains on that. Dohmatob et al. showed 0.1% synthetic contamination is enough to cause measurable degradation. Right now no major dataset (FineWeb, RedPajama, C4) filters for AI-generated content.
What makes this harder to think about: data quality and model performance don't always follow "garbage in, garbage out." I wrote about a related paradox where Qwen2.5-Math trained with deliberately wrong reward signals still improved almost as much as with correct ones: https://ai.gopubby.com/false-rewards-make-ai-smarter-paradox...
Models are simultaneously fragile to recursive contamination and weirdly resilient to corrupted training signals. The picture is messier than either side suggests.
Survey of 65+ papers on model collapse. Key finding from Dohmatob et al. (ICLR 2025): even 0.1% synthetic contamination in training data causes measurable degradation.
No major dataset (FineWeb, RedPajama, C4) currently filters for AI-generated content.
1. Appfigures $2.1M = https://appfigures.com/reports/app-profile/338340235920
2. Watermark bypass = https://www.404media.co/sora-2-watermark-removers-flood-the-...
3. Goldman Sachs $410B = https://www.tomshardware.com/tech-industry/artificial-intell...