batshit_beaver

no profile record

comments

7 dni temudiscuss

I think the issue is that obtaining empirical proof of AI or manual coding being more efficient is very difficult, since true costs and outcomes aren’t known for months and often years. AI _is_ faster at producing short term results though, so naturally, given the state of the industry, everyone is piling both money and time into AI driven workflows and manual work is met with suspicion, if not outright discouragement. Engineers might hate AI, but they need their paychecks.

w zeszłym miesiącudiscuss

Hopefully everyone? Else your job could have been outsourced or replaced by a junior with access to Google and StackOverflow way before LLMs (it just wasn’t due to zero interest rates and proliferation of bullshit jobs in tech companies).

w zeszłym miesiącudiscuss

Humans have goal seeking behavior. LLMs don’t. You could maybe call the combination of LLMs and the RL-based harnesses somewhat “intelligent” in aggregate, but the problem is that it’s not “general” intelligence like these labs want to argue, since it’s by definition only good for the set of problems the RL part has been trained to solve, which is a subset of programming problems.

w zeszłym miesiącudiscuss

> Significantly increased my productivity as a software engineer.

You’re going to have to define productivity as it applies to software engineering. With LLMs we’ve primarily seen the number of PRs over time being discussed as a proxy for LoC, as well as the speed of bootstrapping a small project. None of these have a known correlation with economic output. They just feel good, to the programmer, their manager, or both.

> Using it daily for Chinese-English translation. Significantly better than pre-LLM translation software. Also, great at teaching grammar, nuances, etc.

Yes dealing with language is the one area LLMs are actually designed for. But what’s the TAM for machine translation?

> General Q&A. Like "Googling" but much faster. This is probably the most common use case for me.

And now you’re missing any kind of traceability for the information that you “learn,” since it all gets spaghettified and then recombined into a pile of plausible slop with no attribution. Where before you had to do slightly more work to find the information you needed, now it’s available faster but you’re at complete mercy of literally 3 American companies plus the CCP for the accuracy of that information. Most people somehow seem happy with this arrangement.

w zeszłym miesiącudiscuss

Yes that’s the right source. There would be no recovery in SWE market after the higher interest rates killed it if LLMs had any major impact on SWE employability.

w zeszłym miesiącudiscuss

> The real risk isn't that some 19 year-old vibe coder is going to replace you, it's that there's simply less need for more experienced engineers. The market is shrinking.

That last sentence is verifiably false if you look at SWE job postings and their recovery since 2022.

It’s also a poor take in general, buying very much into the narrative propagated primarily by OpenAI and, especially, Anthropic, who nonetheless continue to hire large numbers of SWEs while paying double the market rate.

w zeszłym miesiącudiscuss

Always has been tbh

w zeszłym miesiącudiscuss

Back in the day, you couldn’t ask stack overflow about your specific business or project. You were forced to build at least some level of understanding of what you were doing on the job or risk your lack of knowledge being obvious (and obviously holding you back).

What we’re seeing now is industrial grade ignorance that can only be observed in in-person or video meetings.

2 miesiące temudiscuss

> We can debate as to how successful we’ve been toward the two goals above, but I think it’s misguided to say that the majority of people think LLMs should produce lower quality code.

Guessing you’re not at FAANG or similar company. For the last 6 months at least there’s been tremendous pressure from leadership (including highly experienced IC engineers) to let AI take the reigns, assumption being that future AI assistants will be able to deal with any level of complexity and tech debt created today.

Given that everyone agrees that reviewing all AI-generated code is impractical (if you let the agents rip at maximum available bandwidth), and that “harness engineering” is at best immature and at worst complete snake oil when it comes to ensuring system stability, maintainability, and quality, I do believe that it’s fair to claim that most engineers are, in fact, supportive of low quality code generated by LLMs.

Fwiw I do see pushback here and there, but only from the lowest rungs on the career ladder - ICs with enough experience to see where this train is headed, but no ability to save it. Management needs to see the results of their policies first, and that will take months or even years to fully play out.

2 miesiące temudiscuss

Can someone explain these complaints about boilerplate to me? What are y’all doing where boilerplate is the majority of your code? Am I the only one mostly writing concise business logic where most lines are important in one way or another?

2 miesiące temudiscuss

If only it was that simple. The reason these inefficient companies continue to exist is due to regulatory capture and monopolistic behavior. Competing with them doesn't just require better efficiency.

2 miesiące temudiscuss

The problem is that organizations are inefficient in such a way that extra output from white collar workers doesn't translate to improved org-wide performance in a positively correlated, linear fashion.

When the org is misaligned, mismanaged, has poor customer feedback loops, bad product market fit, too much bureaucracy, etc etc no amount of AI slop is going to make a meaningful impact on its bottom line. In fact, it will likely do the opposite through combination of exponentially increasing complexity, combined with worker force deskilling, layoffs, and rising token prices. Real bottleneck is and always has been communication & alignment.

It might make the employees _happier_ in the interim though, which, I believe, is what we're predominantly seeing during this AI mania. People fed up with the bullshit jobs of rewriting the same service for the 5th time in 2 years or creating TPS reports weekly just for their manager to throw them directly in the trash are absolutely giddy that they no longer have to do this manually. I think we need to question the economic value of these jobs in the first place, though.

I've worked at big tech prior to LLMs becoming a thing, and consistently saw projects of 20-50 people carried by 2-3 individuals that actually understood what needed to be done. I don't think this ratio will be any better with genAI, and I also don't think that tokenmaxxing has any meaningful correlation with impact. Bullshit jobs (and questionable personal projects) just get done faster now. Yay, I guess.

2 miesiące temudiscuss

Now they're looking at your token consumption, which is even more gameable (and stupid).

2 miesiące temudiscuss

Oh no, we should create a fear mongering blog post and delay the latest IDE version until we have better security in place!

2 miesiące temudiscuss

10x the amount of code or features =/= 10x the speed of software development.

2 miesiące temudiscuss

1. It's unclear why creating more code faster is a good thing. Software engineering wisdom for decades has been that code is a cost, not a product. There are great reasons for that, which haven't changed with the appearance of LLMs.

2. There absolutely are cases where modifying code "manually" is unquestionably faster than prompting an LLM. There are trivial examples for this - eg only an insane person would ask an LLM to rename a variable rather than using an LSP for that. It would provably and consistently take more keystrokes. There are less trivial examples as well, like, you know, having an understanding of your codebase and using good abstractions/libraries within it that let you make large changes to the program's behavior with little boilerplate code.

One can argue that producing a lot of complex changes through an LLM is faster, which I would agree with, but then see point #1. Sustainable software development has up to this point relied on iterative discovery of the right small components that together form a complete, functional, stable system (see "Programming as Theory Building").

There's zero indication so far that LLMs are capable of speeding up the process of creating complete, functional, stable systems. What every org within my career and friend circle is seeing (and research into productivity impacts of LLMs on software development is showing) is the same story - fast prototypes that either turn into abandonware, personal tools, or maintenance nightmares.

2 miesiące temudiscuss

If you read these further, researchers believe this effect does exist, but only insofar as priming the model for the answer it was likely to give anyway and only when queries are in-distribution. If there was actual reasoning involved rather than pattern matching, we would expect to see performance improvements on out of distribution requests. Instead we see longer CoT actually degrade performance on out of distribution tasks.

The fact that common sense, simple logical questions (like should you drive or walk to the car wash) cannot be answered by LLMs simply because they don't appear often enough within pre- or post-training datasets despite CoT is just another indicator of them not performing what we would call reasoning or intent inference or whatever other anthropomorphic behavior we want to assign them. They remain spicy autocomplete with the caveat that the RLHF portion of their training _can_ result in goal seeking and problem-solving behavior... in the narrow set of problems that have been explicitly optimized for in their training.

2 miesiące temudiscuss

Examples:

https://arxiv.org/html/2506.02878v1

https://arxiv.org/pdf/2508.01191

Anthropic themselves: https://www.anthropic.com/research/reasoning-models-dont-say...

They were approaching this from an interpretability standpoint, but the more interesting finding in there is that models come up with an answer that fits their training and context provided. CoT is generated to fit the anticipated answer.

In these studies, there are examples of CoT that directly contradicts the response these models ultimately settle on.

This is not reasoning. This is pretense.

2 miesiące temudiscuss

Right, and then look at any number of research papers showing that CoT output has limited impact on the end result. We've trained these models to pretend to reason.

2 miesiące temudiscuss

GitHub doesn't pay top of market.