> Don't turn a one-off or area-specific comment into a durable memory without my explicit confirmation. You have a history of over-indexing on one-offs, and those memories end up getting cited to override well-tuned skills.
A few people are running highly quantized models with limited context windows. It's still impressive, but not the benchmark level intelligence. Very few people could afford a rig for reasonable local performance at a reasonable quant, at full context size.
The antirez example is 2.6bit quant, 32k context, and few tokens per second... on a ~$7000 MacBook M5 (new RAM pricing).
I dunno. I know a lot of software engineering experts. AI isn't always right, but neither are the people, and it's getting better and better.
Software is one domain where it excels because of structured training data and simulation environments, so I'm well aware it's better here than other areas.
Still there's somewhere balanced between saying every time it's "insufficient or incomplete or outright misleading" and "just trust AI". AI's a useful source of information/reasoning/research, but know you need to validate it's answers for important decisions.
This was copying many screens, composing layouts and many strings. not a "general look and feel" or one off utility string. Others found parts of the code copied (exact variable names). It would take a court to decide, but I think it's pretty clear cut copyright violation. They gave their agent had access to the EE code, and seemingly asked it to produce a copy. It's not a 1:1 copy, but there was a lot of copying. No clean-room attempt was even made.
If I take the first chapter of Harry Potter and write a new ending, it's still copyright violation. They took many parts.
Short segments of popular works sure. Many UI pages with identical layouts and copy, essentially zero chance. The agent had access to the original code at inference time.
It's not economic to run them locally. It's amazing for privacy, and fun hobby. But you're either looking at super slow CPU builds with $10k in RAM, $90k worth of GPUs, or a really quantized model that doesn't compare in quality.
I might build one for fun, but it's not going to change the economics alone. Still exciting it's possible.
My 8yo son plays with this any chance he gets. We've started building watches together as a result (simple "drop in movement mods", but aspirations of building our own movement). The author really made something special.
I think the point is: if every app chooses the best model for their use case, my phone is hosed (disk, memory swap, memory). A good-enough default might be better for the user than the each app having the best possible.
I think that's what they are trying to avoid. If you need on-device intelligence, their pitch was "The model the device already has is best", and if you need something more specific an adapter (aka, a fine-tune/lora) is best.
They were wrong when their on-device model was way behind. They still might be right in the long term.
While multiple app I use might need Gemma 4 E4B, I use dozens of apps and app devs can choose from hundreds of models. A shared cache might reduce size a little when there's overlap, but the core problem still exists. If each app chooses a model disk and memory-swapping explode.
Its probably be better for device manufacturers to bake in a default. I'm not proposing they limit you from using others, but one shared default might be best developer/user experience for 99% of apps.
- Being warm in memory is the single biggest perf speedup you can get, and a default is much more likely to be warm.
- "Best model" is usually "best model for this device" given both RAM and compute. A developer can't test every device but Apple can/will.
- Each model needs to be optimized for the hardware (what's running on ANE, what's running on Metal, what's running on CPU). The default gets optimized.
- If you need custom model, a Lora is probably best (30MB, benefits from all of the above)
You could say the default should be swappable, but that's more a linux ideal than an Apple one so I doubt we ever see that. Plus there are real downsides: intentional or not, prompts end up optimized to the model they are developed for, so swapping the default system model would degrade every app.