This was a great read, thanks. Definitely provides evidence for my fears about vibe coded projects built by people who lack expertise to guide AI well. And if these are the issues you can see, it makes you wonder about what's hard to discover. Security to me is one of those things you want to be thinking about throughout the whole process of building it, not something you tack on at the end.
Some database tests can't be done within transactions. With postgres, I create a copy of the database via WITH TEMPLATE, and each test runs in its own copy of the database. Then it can use or avoid transactions as it pleases, because the whole thing is local to that one tests anyway.
I hate Imgur. Even with the app installed I find it doesn’t work well. I don’t understand why people use it — does it just work for them in a way it doesn’t for me, or are they more tolerant of its terrible usability?
What sort of code are you writing? I find a lot of my stuff requires careful design, refactoring an existing system to work in a new way.
If the code I was writing was, say, small websites all the time for different clients, I can see it being a big improvement. But iterating on a complex existing platform, I’m not so sure that AI will keep the system designed in a maintainable and good way.
But if your experience is with the same sort of code as me, then I may have to re evaluate my judgments.
I'm one of those people who thinks simultaneously that (a) current AI cannot replace developers, it just isn't good enough (and I don't think it's good for it to write much code), and (b) AI is simply an incredible invention and will go down as one of the top 5 or 10 in history.
I've said the same thing as you, that there is a LOT left to be done with current AI capabilities, and we've barely scratched the surface.
Presumably more questions can cover a wider variety of skills/techniques/topics. If the student doesn't know in advance which 10 will be selected, they either have to pray they're lucky, or work diligently on all problems.
The main problem I have with auto-suggestions is that they distract my flow of thinking. Suddenly, I go from thinking about my code carefully, to reviewing someone else's code. To the point where I get a bit stressed typing, worrying that if I go too slow, the suggestion will pop up. As you may guess, I therefore have them turned off :)
I am playing with Zed now though, and it has a "subtle" mode for suggestions which is great. When I explicitly want to see them, I press option key. Otherwise, I don't see them.
Is AI genuinely that good for you all? I can't leave it to its own devices, I have to review everything because (from experience) I don't trust it. I think it's an amazing technological advancement, perhaps will go down as one of the top 10 in the history of our species. But I can't just "fire and forget".
And that's not just because its output is often not the best, but also because by doing it myself it causes me to think deeply about the problem, come up with a better solution that considers edge cases. Furthermore, it gives me knowledge in my head about that project that helps me for the next change.
I see comments here where people seem to have eliminated almost all of their dev work, and it makes me wonder what I'm doing wrong.
Without having RTFA, I'd guess/predict that it will be possible to learn to only do this intentionally, much like we can think about raising our arm without actually raising it.
My own view is an idealist style one, where I think God impresses experiences upon us, and the experiences we have are determined by physical states. On this view, it's impossible to have a religious experience without there being appropriate physical states in place. In other words, agreeing with your conclusion.
There's a difference between reviewing code by developers you trust, and reviewing code by developers you don't trust or AI you don't trust.
Although tbh, even in the worse case I think I am still faster at reviewing than writing. The only difference is though, those reviews will never have had the same depth of thought and consideration as when I write the code myself. So reviews are quicker, but also less thorough/robust than writing for me.
This kind of thing is my main use, boilerplate stuff And for scripts that I don't care about -- e.g., if I need a quick bash script to do a once off task.
For harder problems, my experience is that it falls over, although I haven't been refining my LLM skills as much as some do. It seems that the bigger the project, the more it integrates with other things, the worse AI is. And moreover, for those tasks it's important for me or a human to do it because (a) we think about edge cases while we work through the problem intellectually, and (b) it gives us a deep understanding of the system.
Some people really do act like that, in my limited experience. Trump's opinion changes and lo and behold their beliefs have updated to the same view as Trump's.
That's definitely a reasonable way to think about it. Another though is in terms of social status and ability to direct human labor, in which case most people are not more wealthy.
> we still have potholes to fix, t-shirts to fold and illnesses to cure
Only one of these things interests me. The hype of AI is threatening to kill something I actually enjoy doing. If the hype actualises, I'll likely find myself having to do something I don't enjoy. That being said, if programming can be automated, then probably every white collar job is under serious threat.