I am still skeptical of ai running everything, given how much it changes and quickly, I’ve not quite given over all control. And I try and use it to learn more about things normally I wouldn’t have time for.
The goal of writing the git messages is to slow down and prevent a worst case scenario where work is lost. Given how powerful git is I think giving control over is like doing a sure fire migration without a backup in the way that it can lead to easily preventable problems that are difficult to fix afterwards, it’s just a bad paradigm.
I can appreciate some parts of this, like keeping a painfully detailed record of the changes written by ai, I have this too, but it is separate for the content meant for actual human eyes.
Like others here I find a hard time finding specific evidence or reasons for some of op’s thoughts, but in general it just seems like a recipe for problems when you’re too trusting with ai with all the processes
Ngl I’m doing this right now for a client. Part of my strategy is to write out e2e tests that get a certain baseline of functionality, and then use that as the check for any change that I make to the codebase to make sure it continues to work.
So workflow for a full web app is make e2e tests for all use cases. Then add a very strict duplication checker, and linter, and then just tell the ai to hit a certain duplication limit like 3%, check the linter, and add unit tests to ~95% or greater of the code.
With the right CI and other checks that are deterministic you can really do a lot with a codebase.
My last sentence is definitely aspirational, it is how I try and go about it, but for sure I make mistakes. However your comments about writing spec tests was interesting to me.
Honestly I don’t even write tests manually because of coverage checks. Being that the coverage check is not something easily manipulated, I always tell the ai, don’t ever change configs, and make the coverage pass whatever I set it to, most times > 95%. I just tell the AI, make this coverage pass.
I find tremendous success with this technique, or anytime really I can find an objective way for the ai to test its work.
For sure every time you use ai you’re sacrificing understanding if you don’t plan out and understand how exactly the ai is going to do the work you asked it to do.
The same output that is such a bad thing in this article can also be used to gain context, by making a thorough plan with your ai first, reading through the plan and proposing changes just like you would with a real developer.
You can also use this output to have the ai write a journal as well. The journal can be as detailed as possible and essentially a ledger of all of the changes your ai has made to the code. This allows not only for your teammates reviewing your pr to gain greater context, but also can be used by yourself, or even the ai itself to figure the why behind a particular implementation was done the way it was, far into the future even.
Lastly how many of us ever deploy code without actually checking the feature works e2e? I would gather not many of us do, I don’t, because even though we may have a greater understanding of the code, we can make mistakes in the code or in our logic. And I keep coming back to why would we treat llms any differently? I believe we should be spending our energy thoroughly manually testing a feature to make sure when we brainstormed we actually did get every edge case, and it works well.
> I always wondered why people don't also ask the AI to generate code comments/documentation, summaries of those documentation, overview of the system, and re-review them all for correctness for the changes they asked the AI to do.
I now on all of my projects have an ai journal that stands as a ledger for every change the ai has made, and why it was made. I don’t read it that hard personally because I spend so much time planning with my agent before letting it code. However I have found it very useful in sharing code between people, or having Claude look through the journal to gain context when modifying or adding a feature.
I’m not very familiar with Go, however after looking at the repo I can’t help but notice there is no infra to ensure code quality. Do others see the same thing, because if so that is the real problem
Yes I agree for sure llms write terrible code when left to their own devices, but so do most engineers. Which is why we have so many tools to help keep a certain level of quality. Duplication checks, tests, linters, other engineers.
I find whenever you make an llm repo without these checks, and more, it will write like an enthusiastic junior engineer, wrong and strong. However a junior engineer would be hard pressed to get 95% coverage on a codebase, the ai is more than willing and does it in a few minutes. We can use things like this to our advantage, how many people have ever seen a repo with 100% test coverage? With ai this is very possible, with people not so much.
LLM’s writes terrible code, we know this, but when dealing with humans that write terrible code we have many techniques. We should be using those same techniques to keep the llms honest, but more importantly verifiable.
Weirdly, and i fully think this is just some cognitive bias I don't have the knowledge to name, the ai seems very happy to please me. Like when it gets something done in one shot, it seems very happy to do so.
Well if the spec is incomplete it sounds like you should lower scope for the AI, and then go from there. I wouldn't be too keen to give a junior engineer free reign and expect awesomeness
Completely agree! People tend to forget we are non deterministic too! Yet we are able to write code fine, and fairly reliably by using tools that can help keep us fairly honest.
I think most problems with ai tend to be around can you deterministically test the thing you are asking it to do?
How many of us would never ever show work, without going to check the thing we just built first?
Yeah but that is how many tools just to get it to work, and how much burden on your PC. it just seems simpler to me to just use as few tools as possible to accomplish the goal.
I actually now think ai prompt writing in the IDE is completely overkill nowadays.
IDEs are made for just a human to interact with code. I think the paradigm of forcing these tools that weren’t built for this to do this, is us trying to fit a square peg in a round hole.
Call me old, but don’t put ai in my ide. My ide was made for a human, not an ai. For the established players for sure it makes sense since they already have space on our machines. But for the new ones imo terminal, or dedicated llm interfaces are where it’s at.
If I’m writing code sure suggest the next line. If the machine is writing code, let it, and just supervise properly. and have the proper interface that allows the strength of each
I was just thinking the other day how ai will be way worse than social media in terms of influencing people. Before though I was thinking of times people have been convinced by chat bots they were god or a genius.
Social media was so powerful because it convinced people their world view was popular and correct even if it definitely wasn’t, but at least then it was actually happening somewhere. This is just completely made up, to feed into people’s world view, and they don’t even have to find the perfect figure they can make one up.
We’re definitely going to suffer a lot before it gets better. Interesting times we’re living in.
Eh, for many reasons I am not posting it here. It is a passion project for something and would lead to problems if I post it here. That being said I was trying to share the technique.
The reason for the post is that even without the actual website one should be able to envision the technique and how it may or may not work. Also if you look above recently I added links to the Claude.md for another thing I was working on for a friend that also had to solve this problem.
Just want to give people the tools to use ai well from my own findings
Thank you. Yes took a bit but still way faster than by hand. There are other store pages that are also implemented. This 1 page took me like an hour lol.
Hey, one thing I made with this technique is hartwork.life a simple Wordpress store for a friend. I used open ai to design it for me, and then used the techniques above to get Claude code to implement the proper designs.
I am still trying to learn how to wrangle Claude properly, but I have this Claude.md[1] for that I used to make the website. In particular one of the last rules about using imageMagick for comparison.
I haven’t touched this website in a bit (waiting on client) so now I use playwright mcp for the screenshots and the browser interactions.
I built a few websites, most of them it wouldn’t be wise to place on here. But someone emailed me about this, so I’ll do my best to help I did build https://hartwork.life for a friend with a design from open ai (pre google stitch which is my current preferred tool)
Here is the line from my Claude code to get something like this. Keep in mind I didnt use mcp for playwright with this particular implementation but it is my preferred method currently. Tha
CRITICAL - When implementing a feature based off of an image mockup, use google chrome from the applications folder set the browser dimensions to the width and height of the mockup, capture a screenshot, and compare that screenshot directly to the mockup with imagemagick. If the image is less than 90% similar go back and try and modify the code so that way the website matches the mockup closer. If a change you make makes the similarity go down, undo it, and try something else. be mindful the fonts will never be laid out exactly like the mockup, please use blur at a max of 10% to see if the images are closer matching. If you spend more than 10 cycles screen-shotting and comparing, stop and show the user how similar they are mentioning any problems
The more text the harder it becomes and it’s why we really need the blue because fonts are almost always rendered differently.
Ngl I’m reading this article after having used ai to build a beautiful front end that is pixel perfect.
Yes ai can’t see, it only understands numbers. So tell it to use image magick to compare the screenshot to the actual mockup, tell it to get less than 5% difference and don’t use more than 20% blur. Thank me later.
I built a whole website in like 2 days with this technique.
Everyone seems to have trouble telling ai how to check its work and that’s the real problem imho.
Truly if you took the best dev in the world and had them write 1000 lines of code without stopping to check the result they would also get it wrong. And the machine is only made in a likeness of our image.
PS. You think Christian god was also pissed at how much we lie? :)
GitHub: panda01 medium: khalah.medium.com