HackerLangs
TopNewTrendsCommentsPastAskShowJobs

lowsong

184 karmajoined 2 years ago

comments

lowsong
·6 hours ago·discuss
No. If anything I’m understating the risk.
lowsong
·18 hours ago·discuss
What you're building should be illegal. Educators who use these tools on children should face jail time.

It's hard to overstate the harm of the system you are building. Please understand, there is no way this idea is salvageable or any tweaks or safeguards can make it safe. If you've actually tested this on real children you've already risked extreme harm.

Please, stop.
lowsong
·yesterday·discuss
I don’t think “I know plenty of people who commit fraud anyway so who cares if it’s right” is the ringing endorsement of AI you think it is.
lowsong
·yesterday·discuss
> I think close to 100% of new ambitious projects are going to leverage AI at least to some degree.

Once the free money dries up that number will rapidly tend towards 0%.

> So how much AI usage does it make it an “AI rewrite”?

Any amount.
lowsong
·yesterday·discuss
That "nearly" is doing an awful lot of heavy lifting. It doesn't matter if your AI model is 99% or 99.99% accurate. For a tax return it has to be perfect every time or someone is at best getting a fine or at worst going to prison.

Sure, human error happens too, but humans take accountability. That's why accountants are a regulated profession. Until an AI company CEO is willing to go prison if the output of their model is wrong, these tools are worthless.

But don't take my word for it, head on over to Toot's own terms of service https://toot-books.pages.dev/terms#ai-not-advice

    Toot uses automated and AI systems to generate classifications and reconciliation suggestions. Output may be incomplete or wrong and must be reviewed by you.
    Toot is a software tool. It does not provide accounting, tax, legal, audit, or financial advice, and nothing it produces is a substitute for a qualified accountant or tax adviser. You are responsible for checking Output before approving it or relying on it, and for any decision you make based on it. To the extent permitted by law, we are not responsible for outcomes arising from automated Output you approve without review.
Comparing this to a human book keeper is farcical.
lowsong
·8 days ago·discuss
Ironically it is you who is most at risk from the direction this is taking.

When this house of cards collapses, AI research dries up, and companies pivot to the next hype cycle there will be a generation of people left with atrophied skills and lingering addiction and psychosis. The most flexible will bounce back just fine, but many will never recover from this damage.

Sincerely, I hope you're in the former category.
lowsong
·8 days ago·discuss
Why do all arguments from AI boosters boil down to this same cycle:

A new model is released, AI fans hail it as huge shift in whatever metrics the AI vendor has gamed this time, and all criticism is shrugged off as "not up to date" and met with "try the new model!" Then, once level heads actually put the claims to the test and find it wanting, criticism is met with "you're just not using it right, you have to learn how to prompt/context/loop engineer for best results" until the next model comes out and this argument repeats.
lowsong
·12 days ago·discuss
It's obvious to more people every day that "AI" has never delivered on what it promised. Until now nobody cared because the costs were cheap and it pays to appear cutting edge. Now the price is rising, and the delayed cost of AI-addiction and broken outputs are starting to sting. Not that employers care about the individual harms of course, but a workforce so deep in collective delusion that they can't see the train coming is only useful when the market is delusional too.

Once this snaps, and it will snap suddenly, companies will be climbing over each other to rip out AI as fast as possible. They won't call it that, of course, you're not going to get a CEO on the news talking about how they made a mistake and it was wrong to invest so much in AI tech. But they'll mean it.

The win scenario is that the crash reduces "AI" use to near zero. Spat out into the graveyard of VC hype like blockchain and metaverse before it. Banished to an eternal unlife of scammers running call centre scams, deepfake porn producers, and the occasional "we made AI safe!" startup trying to reignite the bubble again. While companies with their business on the line clamber to announce that they don't use it.
lowsong
·12 days ago·discuss
> we thank ChatGPT, Claude, Gemini, and Grok for tirelessly traversing the combinatorial space of this manuscript.

This is a premise so fanciful this might as well start with "assume a goose that lays golden eggs" that's then been extrapolated into 100+ pages of AI generated text.
lowsong
·12 days ago·discuss
Tried what? Using LLMs to produce software? I'd say the industry has collectively "tried" to do this for a while now and the results have been an unmitigated disaster.

Yes, we should "give up" on using LLMs, but certainly not for a lack of "trying".
lowsong
·12 days ago·discuss
What AI does is the exact polar opposite of "democratizing". We're going from a world where high-quality learning content is plastered all over the internet for free, where open-source projects are desperate for contributors and full of "good first issue" tickets, and where the tools you need to code can be run on any crappy laptop from the past 15 years—to a world where the majority of content about programming is unverified AI slop, where open-source projects are locking down access to protect against an avalanche of drive-by AI garbage, and where you need a $200/month subscription or a $3,000 GPU to run AI models for coding.
lowsong
·12 days ago·discuss
> They don’t even need to be deterministic checks - a shell script that wraps a `claude -p` - and maybe fetches some online resources to stuff into the context - can do your agent’s legal check for it.

No. It can't. If you think that injecting legal text into the context window and appending "make sure the output complies with this law" will solve your problems you have not understood how an LLM works. The kind of checks you are talking about cannot be codified.
lowsong
·17 days ago·discuss
Yes connect the 3rd party company with a vested interest to capture as much of your data as possible with a direct pipe into all your internal sensitive discussions about your product, business, and market positioning in real time.

Has everyone collectively lost their minds? Suggesting anything like this even five years ago would get you laughed out of the room, and actually doing this would be a career ending mistake.
lowsong
·2 months ago·discuss
> outsourcing their decision making and thinking to AI and not really about using AI itself

> I use AI a ton and I'm having more fun every day than I ever did before

With respect, this is what makes me worry.

If someone is a user of AI, can they really tell the difference between "outsourcing" and "using"? I worry that a lot of people will start out well-intentioned and end up completely outsourced before they realise it.
lowsong
·3 months ago·discuss
So?

The research on AI is showing again and again that people that use AI are losing their skills not just in the specific task but more generally. This isn't a "change of skills" it's a fundamental reduction in the skills of knowledge workers.
lowsong
·3 months ago·discuss
> With my newly-acquired superpowers I could knock out the last two pieces in a few days’ work

From the linked post:[0]

> I left an employer that is years behind adopting AI to one actively supporting and encouraging it. As of March, in my professional capacity I no longer write code myself. My current situation was unimaginable to me only a year ago. Like it or not, this is the future of software engineering. Turns out I like it, and having tasted the future I don’t want to go back to the old ways.

It's deeply distressing to watch people fall into AI psychosis. Being smart, accomplished, or experienced is no defence.

After the bubble pops and the industry realises the damage these tools can do to people, folks like the author will have to confront that they were taken in by a lie. Many won't be able to confront that.

[0]: https://nullprogram.com/blog/2026/03/29/
lowsong
·3 months ago·discuss
We're not talking about replacing "writing code", we're talking about replacing your ability to think about the problems critically at all.
lowsong
·3 months ago·discuss
> None of this requires new technology. It requires treating the database as a defensive layer that assumes the caller might be wrong, might retry, and might not be watching the results.

This is one of those takes that is so close to understanding the problem, and then drawing an insane conclusion.

The problem is that AI agents and the code they output is untrustworthy, buggy, insecure, and lacking in any of the standards the industry has developed over the last 30 years. The solution to this is "don't use AI agents", not "change the rest of the stack to accommodate garbage".
lowsong
·3 months ago·discuss
Any company that has become dependant on AI will struggle to survive from here on. By the time many teams realise it'll be too late.
lowsong
·3 months ago·discuss
At least it'll make it easy to audit and replace it all in a few years.