Is there active research in improving the density of models i.e. higher performance at lower parameter counts? Is this something that's even theoretically possible to do with the architecture?
I feel like I can have a much better model for how I can expect the cost of LLMs to change over the next few years. "Build more datacenters" reeks of "just one more lane bro" and because of Jevons paradox I'm skeptical it will bring costs down. Could ASICs and custom processors (ala Groq) move the needle here at all if the model density can't be improved?
I writhe in pain when I try to find a frontend dev guide for a newbie to learn off, and one of the later chapters is some shit like setting up a kubernetes cluster
Considering they rolled back DEI along with everyone else after Trump's second victory, it's difficult to view those previous "values" as anything other than cynical kissing-up to the previous holders of power.
Wow, we used to have a florist/floral nursery around my parent's place LONG ago (like, they had to have ripped it down by the early 2000s) and for years I vaguely recalled it having a somewhat unique looking sign with a big globe lamp and two stacked, rounded-corner rectangular plastic signs. This was almost certainly the brand of sign it was, and the installation of the sign was probably some time in the 1970s. Amazing what small, inconsequential mysteries get solved by a blog article.
Past a certain point there's also a diminishing return on the effort put in to get a raise/promotion and it mostly becomes about managing "up or out" expectations.
I'm about maxed out for development roles compensation wise. By saving most of this compensation and investing it in the S&P 500 and similar indices, I get way more of a return for far less effort. There are days - not months, days - where I'll earn about $7,500 in stock appreciation. The long term trend has me about matching my monthly salary in earnings.
Raises are inflation adjusted so there's no erosion of the underlying capital going into investments.
Why try harder when I'm paid enough to just invest it in the stock market? The biggest problem I have right now isn't how to get a promotion or raise, it's coming up with increasingly contrived excuses to avoid up-or-out and being pushed into more responsibilities.
Another way to view this calculation: if you keep your infrastructure secure and up to date, you (very likely) don't have to pay any ransom in the first place.
If a trans kid calls the suicide hotline and the volunteer suggests they stop wearing dresses to school so people won't bully them, I'm pretty sure the outcome will be far worse than anyone intended. There should be specialists who know how to handle specific kinds of callers.
I wonder if this will lead to a sort of "open sourcing" of music, where the reputation of what one produces will be improved by releasing the raw DAW files/tracks/etc. Even if AI is used to generate the constituent parts of a manually-assembled track, it would still demonstrate to listeners that there was significant human involvement in the process.
Touring, merch, etc will also serve as good "proof of give-a-shit".
This occurred in Florida. This is but one isolated incident, yet, I can't help but notice the two examples that come to mind (this and Christopher Duntsch) both occurred in states with leadership that champion deregulation. Even if deregulation was the ultimate culprit, it seems inexcusable to me that academic/medical institutions aren't self-regulating effectively despite that.
I feel like I can have a much better model for how I can expect the cost of LLMs to change over the next few years. "Build more datacenters" reeks of "just one more lane bro" and because of Jevons paradox I'm skeptical it will bring costs down. Could ASICs and custom processors (ala Groq) move the needle here at all if the model density can't be improved?