AI discussions these days remind me of college when I agreed to "debate" some Christians. You can assign the camps however you want to assume. But the point is that there is not going to be a lot of constructive discussion.
What are the possibilities for adding more high level tasks like "pick up the [arbitrary thing]"? I assume it's 100 times harder to deal with hands and arms in a generic way. But maybe for grippers with two claws it could be more tractable to just output two force vectors per claw or something for the grasp and another two fir the drop. And maybe the SDJ could do reverse kinematics or something.
But one RGB image wouldn't work. So maybe one would need a depth camera.
if you don't find that then you could fake it with personaplex possibly but making another ASR/STT model just listen continuously and transcribe then send to an LLM with function calling. at least that would allow one direction easily.
Are there any open source full duplex models that are out besides PersonaPlex? There was a chinese open one, maybe Fun Audio chat or something, that said it was going to release a full duplex version but I am not sure if it did.
My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.
Well, Taalas has that kind of technology, but the chip they demoed is probably 20-100 times smaller than necessary since it's only an 8b model.
But let's say they could someday scale that up to a much larger model, 72 large chips per wafer and each chip can do 1000 LLM requests at once (Vera Rubin?). So it's roughly the equivalent of an NVL72 rack.
You might be able to serve something like 50000-60000 requests at once. So I think it's more like handling a small city's worth of customers per wafer than the world if you had that.
I believe in less than 5 years we will get to that, but the model size and/or number of agents is going to keep going up also.
I think the profits depend on how well they manage their fleet purchases (or possible sub-leasing?) to get high utilization without overloading or idle racks.
Because accelerators like H200, B300 etc. are highly parallel and designed to run like 200 or maybe 300 sequences at once (depends on the model, just guessing). I assume they finance the hardware and that cost per device or rack is the same whether each unit is handling 10 requests or 150 requests (aside from electricity).
And probably international customers factor into it to get good utilization over more of the night time. And it likely is something that they look at quarterly more seriously than monthly. The biggest risk to profits might be a downturn in business that causes some portion of the financed AI accelerators to go idle or get low utilization for some weeks (that they can't sublease).
Shocking that a well executed AI tutor improves outcomes.
Hasn't computer assisted interactive learning already been proven for years? Why does there seem to be so much skepticism about enhancing it with AI?
Is this just something like, astoundingly slow adoption or poor execution? Being held back by paper textbook makers? Teachers unions dragging their feet?
How can interactive AI driven individually paced learning _not_ be obviously dramatically more effective?
That makes sense in a way, but remember that Meta had previously seen some brief developer glory in the initial Llama release. Going the off-the-shelf route would essentially be giving up on being on the technology frontier in this area, and not monetizing their knowledge assets.
If that is backed up by benchmarks then maybe they should imitate whatever Cursor did. What did they do?
They may eventually have to do that. Or they might be starting with an existing Llama model. Maybe I should have said "huge breakthrough or additional dataset".
My instinct (for better or worse) is usually contrarian. Most people seem very skeptical of what Meta is doing with AI. But, what if, in a way at least, it makes sense?
Maybe Wang has correctly identified that the programming and agentic ability that Anthropic and OpenAI models have has largely come from armies of software engineers creating massive datasets by writing out coding and agentic problems and solutions?
So he told Zuckerberg that. The reason it may be turning into so much friction is that at companies like Anthropic or OpenAI, training engineers were either hired specifically for that purpose or probably mostly handled through contracts with third parties (which again, hired them to train AI). And honestly many of them may be overseas or just happy to have a job in a difficult period. But anyway they wouldn't have very high salary expectations etc.
But Zuckerberg already had 25000 engineers. Why not take say 1/5 of them and get them working on the the dataset? The problem is that those engineers were hired for different prestigious highly paid positions at Meta/Facebook. They were not hired to do tedious grading of AI answers or quiz construction.
But Zuckerberg either has to do this, or spend additional billions on doing it all with external contractors. A third option would be to try to create a massive distillation operation. Or just hope that his engineers could invent some magical new training trick that manifested the agentic and programming skills without the large scale human input.
Or he could release a model trained largely by existing open weights models. Which without some huge breakthrough probably has no chance of surpassing them, so is pointless.
I think most of the substantive criticism of Zuckerberg has been about burning funds. If he gives up the "your job is to grade AI homework now" plan because his engineers refuse, he would need to go through third parties. The additional billions and billions this would cost would create more pressure on the bottom line and shareholder pressure.
It would also give up any potential advantage that Wang may have optimistically sold the operation as, on that using "real" engineers as opposed to lower paid data labelling engineers might result in a higher quality dataset.
At some point, model architectures that don't need such massive datasets or can be created automatically in a way that advances the frontier will probably come about. But right now it doesn't exist.
Further, the way AI works currently, business advantage from AI comes from encoding existing internal intelligence and knowledge. Meta's massive engineering corp effectively has that in their heads. Having them create these datasets is possibly the only way to leverage this knowledge asset in this paradigm.
I guess the problem is it means forcing thousands of people to do a different job from the one they were hired for.
Which implies that you cannot reasonably serve the "disfavored" cohort (self-funded startup or small business). In other words, you have admitted that I was correct in that the business model is deliberately aimed at businesses that likely can absorb surprise expenses, and it's not appropriate for others.
So write "WARNING: your service will go down if you exceed this limit. We cannot provide refunds for this. To avoid outages in the event of unanticipated traffic, do not enter a spend limit."
Why is it that you claim limits are unworkable? If you can track or enforce it (others have been for years) then couldn't you make it an optional field or checkbox?
The short version is it seems like a big "gotcha" that there is no way to limit bandwidth or spending on that or other resources ahead of time, and that might be a deliberate business model that is more aimed at well-funded startups or large companies that are monitoring costs much less closely than an individual or small business.
It's not necessarily too hard to just not dynamically spawn a bunch of machines, but the bandwidth one is going to sneak up on people.
Interesting. How does this compare to Firecracker? Also PhoenixNap looks really interesting. Do you happen to know if Linux software compatibility holds up on Ampere? 80 cores for $400 a month seems pretty good.
I specifically complained to a fly.io staff on here about their "gotcha, b*tch" usage based pricing which they basically copied from AWS, and they stood by it and other people here backed them up. No one is giving me a pile of free money, so I can't risk that kind of thing.