overgard

há 9 horasdiscuss

Right, but my criticism is to the hit-and-run nature of these hype pieces. By the time there's any semblance of what it actually means everyone has moved on but then you have a bunch of people operating under delusions from the hype. I get why OpenAI does it but I wish people would stop upvoting it. Like, hacker news is not a mathematics forum so the only purpose of this kind of thing is hype boosting or polarizing people. I am not looking forward to the "MATH IS SOLVED!" people for the next few days.

overgard

há 9 horasdiscuss

I don't really like these articles, because they seem extremely hard to verify. OpenAI has published a lot of stuff in the past where, upon close inspection, what they're saying is technically true but a lot less interesting or impressive than the headline. Except by the time anyone looks into it, the hype has moved on. It seems like there's maybe a thousand people in the world that can even say if this is good or not?

overgard

há 10 horasdiscuss

Sure; it's an example, I would never rely "Please don't delete" for real important data. However, we're being sold an idea that we should just let the agent do everything, so I think it's important to point out how utterly insane that idea actually is.

overgard

há 10 horasdiscuss

There are options, but they're not great.

I could ban it from using git, but having access to git logs helps it do work so I don't want to go that far. I could probably ban the subcommand, but as a convenience I do like to be able to ask it to make a commit. (I'm not super attached to this, I frequently commit myself just to avoid these issues). It does tend to write good commit messages, so sometimes I ask it to generate the commit message and then just do the operation myself, which kind of sucks in the sense of feeling like reverse-centaur (Cory Doctorow term). (The reason I do that is sometimes it gets confused where if I greenlit one commit, it assumes all future commits are greenlit)

I guess the other option would be to not use "auto" mode, but god there's just no way I want to sit around and it "yes" 50x a session. I'd rather just sandbox it and nuke it if it does something too stupid.

overgard

há 10 horasdiscuss

I entirely agree, although I think this is counter to the narrative that we should just let the agent do everything that the labs want to sell.

There's a really major problem with the concept of human-in-the-loop though, which is just that humans are not built for that that kind of work. It's like all those tests against the TSA where they manage to sneak something through that should have been caught. But the problem is, a TSA agent sees probably like 1000 things they can ignore to the 1 thing they need to look at, and it's easy to just start rubber stamping things.

overgard

há 10 horasdiscuss

Same for me. Not sure if its intentional (do they want to push people towards workflows that will burn more tokens?) or if it's just a side effect, but it's very annoying. I refuse to use tools that dictate how I develop, I'm just going to switch models to less opinionated ones.

overgard

há 11 horasdiscuss

How am I supposed to take this tool seriously if it struggles with the concept of "dont"?

if what you're suggesting is true, then more important instructions like "Don't delete the production database" are a problem. I shouldn't need to consider how to phrase "Don't delete the production database" in a positive manner. Isn't the point of an AI agent that it understands my intent and I don't need to hold its hand? I'm not saying your suggestion is wrong, I just think that suggests a limit to what these tools should be used for if that's the case.

My guess though is that it probably ignores some of the positive instructions too, and the hardcore AI users mostly don't notice because they probably aren't reviewing the work.

overgard

há 11 horasdiscuss

I've seen it with seniors too. The smartest person I worked with (by far!) used to constantly use the menus in Visual Studio (OG Visual Studio, not code) for basically every operation. It was incredibly painful to watch. Watching him debug was a nightmare.

The second smartest guy I worked with couldn't really type properly. (He'd use two fingers). He was still a fantastic coder.

The thing is though, it kind of didn't matter because the value these guys provided was with their incredibly high intelligence, and the friction with how they interacted with tools was more of an issue on the margins than a big deal.

I think for people solving easier problems than these guys (who were working on legitimately hard problems), like, a webdev fixing frontend code, tools might matter a lot because there's less thinking and more navigating and typing. So context matters here a lot. But I definitely don't think you get to be an amazing programmer by CLI mastery (it definitely helps, but it's not a requirement)

overgard

há 11 horasdiscuss

My experience is the more things you add to the list, the worse agents perform (I'm not exactly the first person to notice this)

I actually have a pretty simple set of instructions right now and Claude still regularly messes them up. Like, my first instructions are:

    - Never commit to git without permission.
    - Never sign commit messages

You know what it constantly does? Commits without permissions and signs commit messages! And then I ask it, and it's like "oh, you're right, I have that instruction and I ignored it". If you were working with a junior developer, you'd expect at some point they get it, but with Claude even if I tell it to stuff the instructions into it's memory, it will do it, and STILL mess it up.

Then it gets even more fun, because if I politely correct it in the session, it will understand, but then there's a decent chance it will be completely unable to commit anything in the session going forward. Phd level intelligence!

Anyway, my point is, if you're asking it to review a list of 200 items I guarantee it's quietly messing up on a lot of that list.

overgard

há 11 horasdiscuss

I'm not sure I'd call this a competence issue; it's more of a context issue. If you put talented people in a thick bureaucracy they cease being able to display those talents. I wouldn't view people working in a corporate bureaucracy as having "gone blind" and lost their competence necessarily (especially in this job market). It's like when a hockey team trades a player and suddenly the player doubles their output, because it was a bad system/skill fit/etc, they didn't magically become more talented on the new team, and they hadn't lost their talent on the previous team, they just needed the right system for the talent to show through.

overgard

há 11 horasdiscuss

I don't think your logic is off, but I also think that the FrobnosticatorStudio people have a point. The thing is, yes, the terminal gives you infinitely more capabilities but you probably have like, 20 actual things you do regularly? The learning curve makes it a hard sell when those 20 things are probably all you need. Like, sometimes I'll do something like this if I'm in a terminal and I want to find a build script

    cat packages.json | jq .scripts

And that's useful if I'm in the terminal, but if I'm in VSCode I'll just do

    ctrl-p -> packages.json <enter> -> ctrl-f -> scr

It's actually fewer keystrokes.

I dunno, I've learned that people's workflows are really personal so I'd never tell someone to switch their's, but for me I prefer tools that understand the structure of my project instead of just treating it like text, so IDEs are a preference for me.

overgard

há 12 horasdiscuss

What is with all these people that are so interested in other people's workflow to the point of writing weirdly aggressive manifestos like this. "If you don't do it like me, you're wrong!" I would like to be the first person to state that I don't give a damn about your workflow. Do whatever you want!

overgard

há 12 horasdiscuss

The doom trolling mixed with social media hysteria is creating a huge mental health crisis (not to mention the AI psychosis going on). I was talking to my psychiatrist the other day and she's freaked out. Social media IMO was a huge cause of our current mental health crises, and now AI hysteria is just throwing gasoline on the fire.

overgard

ontemdiscuss

Ok, "most". I think it's a hard sell to go to people that have socialized health care and be like "what if instead of a slightly longer wait, instead you had a system where bureaucrats at insurance providers pull rank over doctors on the treatments you're allowed to receive, and your ability to get healthcare is tied to your employment, and if you get sick you're going to go bankrupt"

Also, the wait time in OUR system sucks too. Try to find a psychiatrist that isn't booked like 3 months in advance. (AI isn't helping with the number of people that need psychiatric services..)

overgard

ontemdiscuss

> But if AI is a race with national security implications, they won’t be fast enough.

I really don't think this is the case. As far as I can tell there are two national security things where LLMs have some utility; mass surveilance (ick!!!) or software security. To me that does not justify a huge infrastructure buildout considering the implications of said infrastructure.

overgard

ontemdiscuss

And yet if you ask any canadian, they all prefer their system to ours.

overgard

ontemdiscuss

Well, if your deductible is like $8000 or something (high deductible), you kind of want to avoid hitting that if possible, because you're out $8000

overgard

ontemdiscuss

In my experience, most rewrites fall more into the realm of "it's easier to write code than it is to read it, and I don't want to read all this existing stuff!" That's kind of the fundamental motivation, and then people couch it in some very plausible sounding technical reasons. (I don't even think people are being dishonest when this happens -- it's easy to look at a codebase and think "this sucks!" because you don't understand the context behind the original decisions.. and writing new code is a lot more fun than maintaining old code) I'm not saying "never rewrite" things, there are valid cases where the original tech stack is no longer relevant or the accumulated tech debt really is too much, but I'm pretty skeptical of most rewrites at this point in my career.

overgard

ontemdiscuss

Honestly, there's just no way I'm giving these tools broad access to my computer. Claude recently started bugging me that I hadn't configured it to be able to access my email, which like, fuck off anthropic. It just seems like a security nightmare and a dangerous level of trust to have in these companies. To me, it's like: here's a folder you can work in, and if you try to do anything outside of it you can fuck off because I'm uninstalling you. I just don't think it's worth dealing with things like potential identity theft or it doing something brain damaged to my data so it can search my inbox or summarize someone's overly-long LLM written email. I know some people run these things in VMs which is smart, but as soon as you give it any sort of network access or access to sensitive data it's just way too big of a risk vector.

overgard

anteontemdiscuss

I don't see this taking off. LLM's produce absolute novels worth of verbose text; and nobody wants to read someone else's conversation with it, and certainly nobody wants to read a multi-year old session log that's 20,000 words long. Also, this is what commit messages are supposed to solve! And the only reason anyone looks at logs are to see what's going wrong, and usually a bisect is easier for finding bad commits rather than doing a deep dive into the specifics.

In the interest of not just being a hater and suggesting an alternative though: if you're implementing a feature (whether yourself or with the agent) have the agent write a spec for the feature and commit THAT. Now you have a nicely organized thing that AI's or people can consume which captures all the intent of the session log with much less noise.

Submissions

Ask HN: Can we talk about AI Astroturfing?

[untitled]

[untitled]

comments