In general, yes. I believe there's a difference between writing a performance review based on your memory/notes of how someone was doing 6 months ago vs getting into the weeds of a sample of PRs/specs and reviewing those from a performance viewpoint.
Author here. This is valid feedback, thank you. This post was originally an internal post. Internally, I have credibility as "the guy that does his homework". That doesn't carry over to the broader internet!
I think you're right that the data presented doesn't prove or disprove the thesis that "customer issues remain broadly stable". I think it lends credibility to the idea that "AI didn't lead to an obvious and immediate slopfest of bugs".
As for the bumps, we've noticed them annually and we haven't yet found a solid reason why. There’s no correlation with customer growth, PRs shipped, or features shipped. It's likely a combination of various seasonal effects in our own hiring and customer onboarding. What isn't in this article: we've more than doubled our customer base in this time period, our customers are more mature and raise bugs more readily, we've got ~60% more engineers, and our product surface area has grown a lot too. I'm yet to land on a satisfying metric that normalizes bug rates in a changing environment.
Author here! That opt out issue is definitely not intentional, I just gave it a quick try and it worked for me in a few different browsers. Feel free to ping me on colin AT ashbyhq.com with any additional details and I'll see if we can get a reproduction.
We only hire engineering managers that are deeply technical. This gives us more ability to work closely with engineers and make sure they're using new tools in the ways we want. It's not perfect. Sometimes, it goes wrong. But, that's why we continuously add tests and other tools to help us spot mistakes.
Author here. 100% agree with your sentiment. When we do performance reviews of engineers we take the time to comb over at least some of the PRs the engineer has written. We flag any patterns around code quality (regardless of whether it came from AI or not). This is on top of any other ongoing feedback we're giving.
E.g. in a recent review I wrote I picked up on an engineer not splitting up React components aggressively enough and over-relying on deeply nested ternaries. In that same review I also commended the engineer for being particularly excellent at explaining complex bugs to our support people and called out one example of a nasty race condition.
I've had burnout. I was 12-18 months in a startup before I realised it was not at all the right fit for me.
I was brought in as CTO to fix some stuff. I fixed that stuff and then realised that how I wanted the business to operate and how the CEO wanted to operate were very far apart.
It all came to a head when somebody got fired and I disagreed with both the decision and how it was carried out. This was just one item amongst a bunch of other things.
This drained me so hard emotionally. I was CTO at another startup for much longer and never got close to this level of burnout. I've talked to someone else about this (talking is helpful) and they had a similar experience. It's not always about the hours or the tenure. 60 hours/week in the right environment is very different to 40 hours/week in the wrong environment.
It took a few years for me to get over this. I took some less complicated jobs with less responsibilities and less personal investment before chucking myself back into the fire. It's going well so far and I'm definitely more resilient as I've been through some pretty dire work shit since then.
Sorry to hear you're having a tough time of it. Getting less personally invested for a while really helped me. I also took on some hobbies that I had total control over and that helped too.