"I'm selling an AI security product and want to establish my brand. I'll post several scare-mongering posts on my blog every week and people like solid_fuel will eat it up because it's what they want to hear."
When running on a GPU, dense models are shaping up to be the best way due to two things:
- Maximum intelligence per VRAM (you dont have much)
- Dense models can benefit from MTP to get an almost 2x speedup in decode (ie, a 27b dense model with mtp decodes at about the same speed as a MoE model with 14b active param model would). This is important because local llm rarely has parallel streams to batch together.
When running on large unified memory like Strix Halo or Spark Dgx, MoE models are usually best:
- You can get similar intelligence as a smaller dense model with fewer active params (to compensate for the slower memory) by throwing ram at the problem.
It's not real. It's like naming your movement "The Good People". It sprouted from the "Rationalist" community, which is even more self-aggrandizing.
Neither has any hope of doing any good for the world as they don't understand evolutionary pressures. They are set up to reward making members feel smart, not accomplishing anything.
And if they ever gain any real power, they will be corrupted immediately.
> Over the past five months, our team has been running an experiment: building and shipping an internal beta of a software product with 0 lines of manually-written code.
This is such a common thing among software engineers nowadays that I was very surprised that OpenAI would open with that line as if it were mind blowing.
But then I saw it was published in February and OP is just reposting it to farm karma.
>You don't need "very much" expert overlap to see aggregate gains at scale, you just need some of it
I'm not sure what you are claiming. Decode is bottle-necked by memory bandwidth. To see a speed up of 2x, you have to ensure each expert weight memory fetch can be used by 2 parallel streams. What exactly is the average factor you are claiming for 5x parallel streams (due to "birthday paradox" factors)? The Birthday paradox isn't really relevant here. It's about coverage, not parallelism.
> Memory for context is an issue, but recent models like DeepSeek V4 use very little of it even at relatively large contexts.
There was only a very brief time it was selling for MSRP (last fall for $2000). Even if you use that as the previous data point, it's only 200% increased.
> I hear "I'm not anti immigrant, I'm anti illegal immigrant" a lot. To which there is an easy solution: increase the number of legal immigrants we allow.
Being "anti illegal immigrant" doesn't have to imply you let in whoever wants as long as they follow some process. You are taking away the agency of the people to select its immigrants.
1) Someone can be against illegal immigration and for legal immigration.
2) That same person's idea about who should immigrate to the country may exclude most or all of the people who are currently immigrating illegally.
It's not like you can only be against illegal immigration because they forgot to fill out some form. Legal immigration has a component of deciding who gets in.