Ask HN: After you ship a feature, what happens to what you learned?
17 comments
My best learnings came from mistakes - mine, my team's, my company's, other companies - but almost always, it was when those mistakes were promptly followed up with a thoughtful incident report.
For example, when I launched what is now Uber One, two days later, I got a call that I broke something. It turned out I had forgotten an edge case in a completely different service. Yes, this was 2016 Uber, 3000 microservices, but I don't think the users who didn't get their discount really cared.
The takeaway I still remember from that today is about always looking for "works by coincidence"; I often ask myself about the preconditions that must be in place for some class of mistake to be impossible.
For example, when I launched what is now Uber One, two days later, I got a call that I broke something. It turned out I had forgotten an edge case in a completely different service. Yes, this was 2016 Uber, 3000 microservices, but I don't think the users who didn't get their discount really cared.
The takeaway I still remember from that today is about always looking for "works by coincidence"; I often ask myself about the preconditions that must be in place for some class of mistake to be impossible.
The framing of preconditions that dissolve a class of mistakes is intriguing. I keep re-reading it, trying to picture how it actually exists day to day: is it something you run in your head when you're in the code, or is it structured into something your team shares (e.g. a checklist or similar)? I haven't had much success closing that gap.
This is very much a shared pain, though my experience is mostly outside the programming world.
I see it as a fixable pattern, but the inertia is strong.
- Lessons learned and retrospectives don't take place as often as they should
- The sessions themselves sometimes get a bit defensive, so the real nuggets get missed
- Even if a good session takes place, it ends up being static data buried in another archive
Taking an example from the construction world, I saw project after project suffer from poor planning and design. Common enough issues to warrant a moment of investigation, but not, then later being passed off as "black swan events". Because who would have thought the ground conditions would be suboptimal near a swamp?
There's something to take from the Health & Safety world. Talk about stuff. An organisation I worked for had a rule that every meeting, irrespective of purpose, had to start with a safety conversation. Something new, a reminder, a good story - just something safety focussed - and a reminder to look in the H&S system for updates and other stories. There's strong correlation between proactive conversations and reduction in safety incidents.
My thinking is that this approach should work across domains. I've done it with teams in the non-programming space with some success. A few minute knowledge-share at the start of stand-ups and regular meetings seemed to reduce the number of times known issues/challenges happened a third time. Just be prepared to lead the conversation with "I was looking through old lessons learned and saw <thing>... good reminder to <action>..." or something.
I see it as a fixable pattern, but the inertia is strong.
- Lessons learned and retrospectives don't take place as often as they should
- The sessions themselves sometimes get a bit defensive, so the real nuggets get missed
- Even if a good session takes place, it ends up being static data buried in another archive
Taking an example from the construction world, I saw project after project suffer from poor planning and design. Common enough issues to warrant a moment of investigation, but not, then later being passed off as "black swan events". Because who would have thought the ground conditions would be suboptimal near a swamp?
There's something to take from the Health & Safety world. Talk about stuff. An organisation I worked for had a rule that every meeting, irrespective of purpose, had to start with a safety conversation. Something new, a reminder, a good story - just something safety focussed - and a reminder to look in the H&S system for updates and other stories. There's strong correlation between proactive conversations and reduction in safety incidents.
My thinking is that this approach should work across domains. I've done it with teams in the non-programming space with some success. A few minute knowledge-share at the start of stand-ups and regular meetings seemed to reduce the number of times known issues/challenges happened a third time. Just be prepared to lead the conversation with "I was looking through old lessons learned and saw <thing>... good reminder to <action>..." or something.
I have my own personal formal debrief process for larger projects: https://github.com/EternityForest/THUNDERWARS
Usually if I notice anything of interest to others, it just winds up in a group chat.
Usually if I notice anything of interest to others, it just winds up in a group chat.
My brain stores it away in a magic database that cannot be queried in any useful way. If I say say "brain, give me a list of learnings from projects you have worked on" - it returns an empty set. But if I'm reviewing someone's code, my brain suddenly pipes up and says "hey, I smell bad code, this is poorly designed and will not perform well - just like that time 7 years ago in a previous role when you smashed your head on the desk for hours trying to get to the bottom of that legacy spaghetti code slowdown"
> Or we ship a feature and never circle back to check whether it's used the way we intended, so we keep shipping more on the same untested assumptions.
It's not on you to track the whole lifecycle of a feature, that belongs to PO+UX.
Analytics and A/B testing are part of good feature planning and should be considered well before things get to dev.
It's not on you to track the whole lifecycle of a feature, that belongs to PO+UX.
Analytics and A/B testing are part of good feature planning and should be considered well before things get to dev.
I'm with you on the usage half: it's planning's job, and analytics/A-B play an important part there.
But I think we're talking past the point that a lot of what you learn building something never show up in any dashboard, because it isn't about what users did, it's what you worked out in the code. That "oh, this whole subsystem quietly assumes X, and the thing I'm adding will break that" moment. By default it lives in the heads of whoever worked on the code and I think is often gone by the next cycle.
I didn't mean to get into who should own what, but I'm trying to ask: the last time you had an insight you felt should be considered next time, did you have a place to put it? Do thoughts like that reach planning, or just dissolve?
But I think we're talking past the point that a lot of what you learn building something never show up in any dashboard, because it isn't about what users did, it's what you worked out in the code. That "oh, this whole subsystem quietly assumes X, and the thing I'm adding will break that" moment. By default it lives in the heads of whoever worked on the code and I think is often gone by the next cycle.
I didn't mean to get into who should own what, but I'm trying to ask: the last time you had an insight you felt should be considered next time, did you have a place to put it? Do thoughts like that reach planning, or just dissolve?
on paper, i write all the major and important parts, words, functions, system, architectures, orchestration, file tree, journals and diaries, i am not an expert coder at all, pc engineer 25 years, and now the past couple years have been using ai , chatgpt and cursor to play with and learn what is possible. when i write something on paper with red or blue texter , and then have that paper in my view while i am working. helps me every single time. helps me not drift. keeps me anchored. these papers are my light house's .and i am captain of my ship.
I often kept a personal "notes" file while I was working on a project/feature. I now think that what I should have done is check that file into the source tree, at the top level directory for whatever feature I was doing.
I like the "check it into the tree" instinct. I've tried tracking observations and lessons in README files across a monorepo, but I felt they went stale almost immediately. Maybe your more focused files would work better. But checked in or not: have you found yourself reopening those earlier notes when you start the next feature? Or do they mostly just sit there?
Well, see, I've never actually done it. I decided that it would be a good idea. And then I retired...
With two decades at the same company, I mostly see this at an organizational level. New management comes in and is expected to fix things, but they try the same things that have already been tried and it fails for the same reason if failed the last 3 times it was tried. Lessons aren’t learned because there is a lack of continuity.
Leadership isn’t allowed to learn from their mistakes, they get replaced. It appears this is common, so ultimately, the industry has these people “transforming” organizations, running projects, or consulting, who just bounce around and implement. They never stick around to see the changes run. They aren’t there for the failure and therefore can never learn from their mistakes. As soon as the project is far enough for them to declare success, they start using that as the springboard into their next position. They get new job based on their “successful” implementation… and then the whole thing falls apart, because they were never worried about how it would run for the next 10 years, just about making it look good long enough to get their bonus and get out. This is how they can avoid ever having to answer for mistakes and can delude themselves into thinking everything they’ve done actually worked.
The other issue I see, that is semi-related, is not giving things long enough to work. Massive cultural shifts take time. If they don’t catch on fast enough, they are abandoned for something else. This teaches everyone that it’s not worth getting on board with these attempts. For something to succeed, it now needs to be the norm for long enough to overcome the lack of trust this pattern created. We got close with agile, they but a lot of weight behind it and management was very consistent. Then when we were on the verge of really turning the corner and running smooth, even with resistant team, they did a re-org and the new guy threw it away and replaced it with nothing, destroying all that progress and teaching people, yet again, that it’s not worth adopting these models.
Leadership isn’t allowed to learn from their mistakes, they get replaced. It appears this is common, so ultimately, the industry has these people “transforming” organizations, running projects, or consulting, who just bounce around and implement. They never stick around to see the changes run. They aren’t there for the failure and therefore can never learn from their mistakes. As soon as the project is far enough for them to declare success, they start using that as the springboard into their next position. They get new job based on their “successful” implementation… and then the whole thing falls apart, because they were never worried about how it would run for the next 10 years, just about making it look good long enough to get their bonus and get out. This is how they can avoid ever having to answer for mistakes and can delude themselves into thinking everything they’ve done actually worked.
The other issue I see, that is semi-related, is not giving things long enough to work. Massive cultural shifts take time. If they don’t catch on fast enough, they are abandoned for something else. This teaches everyone that it’s not worth getting on board with these attempts. For something to succeed, it now needs to be the norm for long enough to overcome the lack of trust this pattern created. We got close with agile, they but a lot of weight behind it and management was very consistent. Then when we were on the verge of really turning the corner and running smooth, even with resistant team, they did a re-org and the new guy threw it away and replaced it with nothing, destroying all that progress and teaching people, yet again, that it’s not worth adopting these models.
That reads like the same pain from higher up, yeah. It's the org forgetting instead of the person, but same essence. I've seen hints of this myself, and I have found no tractable solutions other than packing up and leaving when the winds change direction in high management. Thanks for sharing.
[deleted]
Does this resonate? I'm fairly sure we all hit this, but I can't tell how many people see it as a fixable pattern versus just part of the job.
- After you finish a feature or project, where does what you learned actually go? Is it your head, an issue comment, a doc or wiki, a retro, somewhere else?
- When you start something new, do you explicitly pull in past learnings, like copy, link, reference, discuss it? Or is it all from shared memory?
- Got an example where remembering something earlier would have saved you real time? What happened?
Trying to work out whether this is a shared pain or just mine. War stories very welcome.