HackerLangs
TopNewTrendsCommentsPastAskShowJobs

FiberBundle

957 karmajoined 8 lat temu

Submissions

SlopCodeBench: Benchmarking How Coding Agents Degrade over Long-Horizon Tasks

arxiv.org
2 points·by FiberBundle·4 miesiące temu·0 comments

comments

FiberBundle
·przedwczoraj·discuss
I desperately hope that the Andrew Kelley style of software engineering will survive all of this; that users will continue to value quality and not be content with slop. This, of course, presumes that products built fully by agents will produce sub-par quality in the future. If they will be able to manage to glue all of this slop together without the project collapsing in on itself, none of this will matter. I just hope that this isn't the future of the industry.
FiberBundle
·28 dni temu·discuss
Most of these republicans/libertarians only want the government to leave them alone. They don't care when a company they aren't affiliated with is regulated. You can see Marc Andreesen celebrating the government's decision on Anthropic. Similarly, when Silicon Valley Bank went bankrupt, libertarians such as David Sacks were loudly calling for government bailouts. It's just hypocrisy all the way up.
FiberBundle
·2 miesiące temu·discuss
Another potential reason, not mentioned in the article, is that open source models obviously pose the biggest threat in the labs' ability to monetize their tech. Anthropic especially seems to be very anti open-source. If frontier models start to plateau and don't have capabilities that truly differentiate them, nobody will pay what the labs would want to charge. Posing the tech as a danger is a way for them to make the government regulate open source models.
FiberBundle
·3 miesiące temu·discuss
Looks as if AI sucks at frontend tbh.
FiberBundle
·3 miesiące temu·discuss
To me it makes absolutely zero sense that they would decide to not release the model to the public because of the effects that it would have due to its exploitation capabilities. Previous models were also capable of providing harmful information, yet that wasn't a problem, because models can actually be effectively censored using RHLF. So what is preventing Anthropic to simply forbid the model from letting people vibe-code exploits???
FiberBundle
·3 miesiące temu·discuss
This could also be a marketing strategy. Make your models perform worse towards the end of a model's cycle, so that the next model appears as if more progress has been made than there actually has been.
FiberBundle
·3 miesiące temu·discuss
This is the dumbest take there is about vibe coding. Claiming that managing complexity in a codebase doesn't matter anymore. I can't imagine that a competent engineer would come to the conclusion that managing complexity doesn't matter anymore. There is actually some evidence that coding agents struggle the same way humans do as the complexity of the system increases [0].

[0] https://arxiv.org/abs/2603.24755
FiberBundle
·3 miesiące temu·discuss
This is like saying that innovative medical drugs could be sold at a profit if only there was no patent protection and the innovative companies would still invest in R&D. Yes, on a token level pure inference costs might be profitable, but the frontier Ai labs will surely have to recoup their R&D investments at some point.
FiberBundle
·4 miesiące temu·discuss
To help readers view the content of the article with scepticism, given the results the advice in it seems to produce.
FiberBundle
·4 miesiące temu·discuss
Pine Town [1], the "whimsical infinite multiplayer canvas of a meadow", also looks like pure slop.

[1] https://pine.town/
FiberBundle
·4 miesiące temu·discuss
I'm kind of sceptical about the altruistic motives here. Giving this to open source maintainers also solves the problem of identifying high quality feedback/rewards for their rlvr models. With everybody using Claude code it might be difficult for them to find a robust way to tell apart good reward signal from mediocre or below average feedback.
FiberBundle
·5 miesięcy temu·discuss
Lex as in Lex Fridman? I'm baffled that anyone would say that Lex Fridman is a better interviewer than Dwarkesh. Fridman is the one who continuously rambles some incoherent nonsense and completely lacks the intelligence and knowledge to ask reasonable questions.
FiberBundle
·5 miesięcy temu·discuss
How do you see the solo bootstrapping landscape going forward? In what ways do you see agentic coding changing things?
FiberBundle
·6 miesięcy temu·discuss
The primary value is accrued by the AI labs. You pay hundreds or thousands of dollars a month to train their AI models. While you probably do increase your productivity saving time typing all the code, the feedback that you give the agent after it has produced mediocre or poor code is extremely valuable to the companies, because they train their reinforcement learning models with them. Now while you're happy you have such a great "assistant" that helps you type out code, you will at some point realize that your architectural/design skills really weren't all that special in the first place. All the models lacked to be good at that was sufficient data containing the correct rewards. Thankfully software engineers are some of the most naive people in the world, and they gave them that data by actually paying for it.
FiberBundle
·6 miesięcy temu·discuss
What's funny is that a lot people on Twitter claiming that they can just vibe-code away their SaaS subscriptions are building SaaS themselves.
FiberBundle
·6 miesięcy temu·discuss
Programming without flow state. Nice.
FiberBundle
·6 miesięcy temu·discuss
It just boggles my mind that anyone would use something like this. Why would one send their data to some unknown company that internally likely just delegates the work to one of the big AI labs?
FiberBundle
·8 miesięcy temu·discuss
This is inconsistent with people like Hassabis or Sutskever giving time frames while also saying that LLMs won't get us to AGI.