So, another victory for the LLM. We were told by project maintainers that AI generated pull requests for vulnerabilities would be blocked. Looks like humans take another L. We have to get out of the way.
They do if they are a reasoning-variant. That doesn’t necessarily mean it actually needed to reason for many questions, your prompt + regular context could be enough to get a good answer compared to prior models where you’d absolutely have to put it into a reasoning-loop to get an accurate answer.
It’s on by default, in a way. You can probably prompt these models with “and don’t reason about it, just give me the answer” and probably get a comparably good response without it using reasoning tokens for many things.
A) Context and prompting cuts the search space for next token generation. That’s pretty useful, as you mentioned.
B) The other use of context is that it introduces entirely new information via RAG
B will never go away (as others pointed out). A, well that’s just something we’re all going to keep getting surprised at. We’ll barely give it any direction or context and the newer models will simply find the happy path.
The author is kind of suggesting that their context wasn’t really necessary to get the happy output, I think.
Chain of reasoning is a lot of context to guide token generation, but we simply see that newer models don’t need that context to get to the answer. I’m mostly reiterating this because there’s a hot take here, and that is this agentic stuff may be waived away by magic frontier-llm wand , all of a sudden.
Look man, I’ve got a MMO that I’m working on that’s set in 2014 where everyone is a programmer in SV (might call it World of Legacy). It’s a period piece. I NEED as much blog training data of this type so that my NPCs can talk in a historically accurate way (god bless Medium.com, a historical treasure trove of a bygone medieval era).
It’s gonna be a living breathing world, you see. You’re going to be like “omg, this game even accurately captured the blog posts, woah”.
Edit:
This whole blogging-about-cloud-AI genre is just weird and irresponsible now)
I sincerely never considered it was a whole genre.
Right, the one person (Neo?), has to be able to verify the output. Most of us can’t replace the accountant because we can’t really … replace the accountant. We can replace the accountant’s work, but we can’t replace his/her other work which is knowing if the accounting is right. However, an accountant can replace N other accountants.
One programmer can know if the program is right. So N additional programmers, in that case, can truly be replaced.
I can know the SQL query came back right, I can know the drop down menu looks right, you know what I mean? I can even know if it’s any good. [Person .., ] of Type Y (let’s say Y is “programmer” for now) can be reduced down to 1 in that case (we no longer need the full array of people of a certain type anymore, just the one). Over time we’ll see all different “types” get collapsed down.
Isn’t this just a form of the bitter lesson? Our attempts to make engineered context and agents will simply be made obsolete with bigger and better models. Those transcripts are probably extremely useful for lesser capable models, and near unnecessary for frontier ones, maybe?
When I think about the typical team composition needed to build a typical app in a lot of companies, well, actually what do I think about? I think, okay, that’s how many people we need to build a typical app - some number. That’s what we should all think … like, what is that “number” of people we need?
So, today, if I think really really really, and I mean, really fucking hard about that “number”, I really really really come up with a number close to 1 (we need one person to type in the instruction at least, maybe two? Maybe three? Definitely not a team … no not that at all).
That means, for everything that we build today, before we even build it, we’ve already mentally replaced people. People be replaced yo, in your mind, in your company, in your reality. That’s it.
It’s a replacement, not a tool. And if you reaaallly think a bit more, you’ll see that it is truly the stupidest possible thing if in your scoping of what needs to be built, you didn’t scope out people. You’d have to be crazy not to scope out people.
Trust me, I never thought we’d get to this point either. I thought it was going to be an “assistant” too.
—-
The layoff numbers still look “normal “ to us. 75k here, 20k there, all numbers we’ve heard before. It hasn’t even really happened yet, the true numbers should be in millions. I didn’t believe the number “trillion dollar company” until, BAM, here’s a trillion dollar company.
Best we can do is kinda get used to the numbers, like the number 0 on one end (how many people you need), and some number N (in millions) that we won’t need anymore. We’ll just slowly get used to these new numbers.
The RTX GPU laptops run very hot. Even though they are pound for pound better, it’s just runs too hot for local llm usage for me at least. Prefer Macs for this. A lot of AMD cards also run cooler. I wonder if undervting would help with smaller models and heat.
Sisters, a 8B local model that hallucinates is better than that 50bn in sweaty VR shit they made. It is what it is, slap a monthly fee on the 3B model and move on, I suppose.
The problem with certain intellectual pursuits is that it becomes its own little sub culture with its own little sub culture celebrities.
You see, High School never ended. Things can still get lame in the “real world”. The “geeks” need to shut up and go back to the geek table and be more humble. The whole lot of us have demonstrated limited ability on how to be decent.
To quote Rick James:
”They should have never given you developers money. Fuck your Ping Pong table, fuck. Your. Ping. Pong. Table!”
Getting to the top page is already a systemic flaw that originated from the genesis of this place. The original sin is that the people that run this place have a priority to promote “something” first and foremost.
Getting “to the front page” is a whole other ethical issue, but we have no oversight over the watchmen.
Just trust. So trust requires trust , how much of this place is a giant advertising campaign?