Is it though? Do we still have the expectation that LLMs will eventually be able to solve problems they haven't seen before? Or do we just want the most accurate auto complete at the cheapest price at this point?
They're increasing reps and therefore total load. That's still a form of progression ('pushing yourself'). This style will slightly favor hypertrophy gains over strength gains.
At 40 I recently made this switch in style as well. The weight was getting so high that my anxiety was causing a mental aversion to working out altogether. Consistency is really 95% of exercise so I think this is a reasonable trade-off.
That said, I understand where you are coming from. There's something to be said about facing the fear of the weight head on. I've already done that in my younger years though. I'd much rather avoid injury and get 80% of the benefits.
Extensive tailwind training data in the models. Sure there's something more efficient but it's just safer to let the model leverage what it was trained on.
Professional legal services seem to be picking up steam. Which sort of makes sense as a natural follow on to programming, given that 'the law' is basically codified natural language.
It is unclear. Everyday I seem to read contradictory headlines about whether or not inference is profitable.
If inference has significant profitability and you're the only game in town, you could do really well.
But without regulation, as a commodity, the margin on inference approaches zero.
None of this even speaks to recouping the R&D costs it takes to stay competitive. If they're not able to pull up the ladder, these frontier model companies could have a really bad time.
While it doesn't seem we can agree on a meaning for AGI, I think a lot of people think of it as an intelligent entity that has 100% agency.
Currently we need to direct LLM's from task to task. They don't yet posses the capability of full real world context.
This is why I get confused when people talk about AI replacing jobs. It can replace work, but you still need skilled workers to guide them. To me, this could result in humans being even more valuable to businesses, and result in an even greater demand for labor.
If this is true, individuals need to race to learn how to use AI and use it well.
They know that LLMs as a product are racing towards commoditization. Bye bye profit margins. The only way to win is regulation allowing a few approved providers.
That would be true in a monopolistic market. But these frontier models are all competing against each other. The incentive to 'just work and get shit done fast' is there as they each try to gain market share.
Not required of course. This move to finally move some of his cash horde into equities could be a result of the US governments recent announcement that they're ending their quantitative tightening policy. This likely signals a move back to a period of quantitative easing which could cause assets to continue to inflate while the value of the dollar continues to erode.
Basically he could just be trying to protect the value of his dollars by putting them into a very stable company that also has exposure to the upside of AI.
I'd call text the most versatile interface, but not sold on it being the ultimate. As the old saying goes 'a picture is worth a thousand words' and well crafted guis can allow a user to grok the functionality of an app very quickly.
I think you make a very interesting observation about these bubbles potentially being an inherent part of new technology expansion.
It makes sense too from a human behavior perspective. Whenever there are massive wins to be had, speculation will run rampant. Everyone wants to be the winner, but only a small fraction will actually win.
Good point. But can the models even behave that way? They depend on probability. If they put a greater weight on novel/unexpected outputs don't they just become undependable hallucination machines? Despite what some people think, these models can't reason about a concept to determine it's validity. They depend on recurring data in training to determine what might be true.
That said, it would be interesting to see a model tuned that way. It could be marketed as a 'creativity model' where the user understands there will be a lot of junk hallucination and that it's up to them to reason whether a concept has validity or not.
I guess I'll take the other side of what most are arguing in this thread.
Isn't it a great thing for to us to collectively allow LLM's to train on past conversations? LLM's probably won't get significantly better without this data.
That said I do recognize the risk of only a handful of companies being responsible for something as important as the collective knowledge of civilization.
Is the long term solution self custody? Organizations or individuals may use and train models locally in order to protect and distribute their learnings internally. Of course costs have to come down a ridiculous amount for this to be feasible.
If your work was truly novel, wouldn't the odds of it showing up in later models be extremely low given that these are probabilistic?
In a sense these machines are outputting the aggregate of the collective thoughts of the commons. In order for concepts to be output they have to be quite common in the training data. Which works out kind of nice for privacy and innovation because by the time concepts are common enough to show up through inference they probably deserve to be part of the public knowledge (IP aside).
Google is the only USA based frontier lab releasing open models. I know they aren't doing it out of the goodness of their hearts.