How to evaluate a product roadmap, for engineers(stephen.fm)
stephen.fm
How to evaluate a product roadmap, for engineers
https://stephen.fm/how-to-evaluate-a-product-roadmap/
32 comments
Unfortunately that is a common expectation for any engineer higher than junior level. It's a strange attempt to get engineers to own vertical slices of the company, as if they were managing their own business within the business.
The problem is that product owners and managers are not learning software engineering to better communicate, design etc., This is a one way street, engineers have more and more non-technical responsibilities until they are barely coding and then are forced to manage a team of junior engineers to do the code for them.
The problem is that product owners and managers are not learning software engineering to better communicate, design etc., This is a one way street, engineers have more and more non-technical responsibilities until they are barely coding and then are forced to manage a team of junior engineers to do the code for them.
Cue ‘Developer Hegemony’, a great book on this topic. Yes, as an engineer, you should be interested in vision/strategy/product. Otherwise you’re just executing, and the opportunists will get promoted ahead of you. The book explains it much better than me.
Response from Amazon/Kindle:
> This title is not currently available for purchase - Have you moved recently?
Apparently this book did not pass the Dutch or Amazon censor. Makes me extra curious.
> This title is not currently available for purchase - Have you moved recently?
Apparently this book did not pass the Dutch or Amazon censor. Makes me extra curious.
It was published by the author as a DRM-free ebook in all formats. You can get it from, say, Bol and convert it. Or directly from Daedtech’s website.
Also engineers would need to manage their technical roadmaps - consisting of tech debt and other infra stuff. Every engineer should know the basics of product atleast
Personally I've always found "risks" in product roadmap tend to not be great. There's often no risks for 80 percent of the project then at the end we'll discover blockers and risks from teams who weren't (or weren't thoroughly) consulted. It's really hard to know and communicate with all the stakeholders in these complex organizations.
Most of the time risk analysis as it's done today is completely useless, as we tend to be pretty bad at risk assessment in general, and that is even worse in most organizations. Barry O'Reilly has a great talk on the topic somewhere on YouTube
You are right, as a civilisation we have no idea how to do risk assessment, and often when sh*t hits the fan, it is something we can't do anything about. At the current rate it is better to NOT do any risk analysis, and just be prepared for the worse case scenario.
"And then an asteroid will hit us and we will become extinct."
[deleted]
Most roadmaps I've seen are usually a wishlist of features written with no consultation with the people who have to implement it, and shared with customers already.
I've seen a lot of bad roadmaps. They usually have multiple projects, all set as the number one priority, all relying on the same fixed resources.
> But the caveat here is, the CPO and leadership team should do this.
In my experience, if you don’t have a smart lead engineer who is working directly with the product people and is involved with the higher level roadmap then life is more difficult for engineers. There needs to be someone advocating for engineering needs and setting the correct expectations about what reasonably can be built and at what cost. If engineering isn’t involved, then there ends up being this toxic one-way relationship where engineers are just receiving marching orders from someone else who might not know what is impossible or who might not have the technical vision to dream of what might be possible.
In my experience, if you don’t have a smart lead engineer who is working directly with the product people and is involved with the higher level roadmap then life is more difficult for engineers. There needs to be someone advocating for engineering needs and setting the correct expectations about what reasonably can be built and at what cost. If engineering isn’t involved, then there ends up being this toxic one-way relationship where engineers are just receiving marching orders from someone else who might not know what is impossible or who might not have the technical vision to dream of what might be possible.
This is just systems engineering 101. There are volumes of material written on this topic including process, documentation, quality and organizational requirements. There is no need to reinvent the wheel. The product “roadmap” is the output of a good systems engineering approach applied to a timeline by a savvy, business- oriented project management team.
I would think the roadmap is fully owned by managers just like the code is fully owned by engineers. Do we ask the product owners to be aware of the code the engineers are writing, just in case there are inconsistencies with their specification? Of course not.
This is very idealistic and removed from reality. Most companies are so messed up that there is no coherent strategy, so you cannot require a single product manager to have an excellent roadmap, when the conditions for creating one is not there. You also may be working on a product or area that is far below the high level strategy, so it's goals would be different.
If an engineer really goes and thinks they should evaluate a roadmap like this, that's toxic.
If an engineer really goes and thinks they should evaluate a roadmap like this, that's toxic.
It may be ideal to have all these items checked off. I think a productive way to look at this is "how many of these items does the roadmap check"?
If the answer is "very few" then that might be an early warning sign you're working for a product or org that is headed for serious problems. As a technical IC who can't hope to solve those large scale management problems, this can be useful a red flag. Don't wait for shit to hit the fan to leave.
I have seen a large scale failure of this kind and the items listed here line up very well with some of the root causes I observed.
- Is the roadmap flexible or iterative? The roadmap was hard, aggressive business targets.
- Are the roadmap initiatives scoped and prioritized based on evidence? The roadmap initiatives were derived by working backwards from business targets and then evidence was found after the fact.
- Does the roadmap identify major dependencies or risks? Many risks were identified much later because techincal teams were not part of input to initial planning.
- Does the roadmap feel aggressive but achievable? Aggressive but not physically possible.
- Does the roadmap take on appropriate risk? No, there was multiple possible independent points of failure.
Anyways, if you ever see this kind of product culture I suggest running for the hills unless you like having your time wasted. And if you are a technical manager I hope you push back like hell when presented with this situation.
If the answer is "very few" then that might be an early warning sign you're working for a product or org that is headed for serious problems. As a technical IC who can't hope to solve those large scale management problems, this can be useful a red flag. Don't wait for shit to hit the fan to leave.
I have seen a large scale failure of this kind and the items listed here line up very well with some of the root causes I observed.
- Is the roadmap flexible or iterative? The roadmap was hard, aggressive business targets.
- Are the roadmap initiatives scoped and prioritized based on evidence? The roadmap initiatives were derived by working backwards from business targets and then evidence was found after the fact.
- Does the roadmap identify major dependencies or risks? Many risks were identified much later because techincal teams were not part of input to initial planning.
- Does the roadmap feel aggressive but achievable? Aggressive but not physically possible.
- Does the roadmap take on appropriate risk? No, there was multiple possible independent points of failure.
Anyways, if you ever see this kind of product culture I suggest running for the hills unless you like having your time wasted. And if you are a technical manager I hope you push back like hell when presented with this situation.
I am a "Group Product Manager" and at least on skim, this article doesn't seem crazy. I would expect myself and folks on my team to articulate these things - or call out when we don't have something for a good reason (eg we're making a speculative bet.)
The thing that I don't expect is that the roadmap would stay static. Reality changes over time, you encounter new customers and opportunities, learn what's harder or easier than you thought, etc. So I would say "this is what we have today" and if that evolves somewhat even a few months down the line as we learn more, great. But at any given point, you should be able to articulate your best understanding in something that does hit against these checkpoints.
The thing that I don't expect is that the roadmap would stay static. Reality changes over time, you encounter new customers and opportunities, learn what's harder or easier than you thought, etc. So I would say "this is what we have today" and if that evolves somewhat even a few months down the line as we learn more, great. But at any given point, you should be able to articulate your best understanding in something that does hit against these checkpoints.
Right, so you're claiming that everything in your roadmap is scoped? That's what the article says to require.
At least on some level, sure. Meaning if we have something far out and we just say "build for x and we'll figure that out closer to the date" that's fine.
But if it's say in the next few months we better have a good stab at what the MVP or the next iteration and the initial market could look like.
But if it's say in the next few months we better have a good stab at what the MVP or the next iteration and the initial market could look like.
I agree with this. If you're working in now/next/later terms, you should have a more detailed definition of scope as things move from 'later' to 'next' and especially to 'now', but things shouldn't get onto the roadmap without at least an idea of what kind of lift you're planning to take onto your schedule.
Roadmaps come in many different flavors depending on the company/culture/processes/etc. For an org where roadmaps are more formal/solidified - a healthy level of "product discovery" should be happening before items are put on the product roadmap. We do it this way at my current company (am a Product Director, leading multiple Product teams). When items are put on the roadmap, scope may not be 100% final, but there is a relatively high level of certainty for what's in/out of scope based on customer & business value, as well as clear prioritization based on value/effort. And for larger scope/high priority items, they've already been aligned to a good extent across key stakeholders & partner product/engineering teams before formally being put on the roadmap.
Scoping is pretty much bs until the thing is mostly done anyway.
Author here, thanks for sharing your feedback. To your point, product is a high-stress, demanding role, and in reality, you might not hit every item from this list all the time. This ambitiously attempted to define greatness from when I've been fortunate to get there, see it, or be part of it. Getting to greatness can be done; it's just hard to achieve. I think it deserves that caveat up front, and I will update it.
I also agree that low-level teams might be very far removed from strategy, especially in large orgs. However, that team should have a clearly defined goal or mandate instead. So much of the product is context-driven, and the first few drafts were 3-4x longer to account for these differences between organizations, hierarchy, teams etc. I removed most of that to focus more on basic principles because this guide was starting to look like it was written by Charlie Kelly chasing Pepe Silvia accounting for all the edge cases :)
I also agree that low-level teams might be very far removed from strategy, especially in large orgs. However, that team should have a clearly defined goal or mandate instead. So much of the product is context-driven, and the first few drafts were 3-4x longer to account for these differences between organizations, hierarchy, teams etc. I removed most of that to focus more on basic principles because this guide was starting to look like it was written by Charlie Kelly chasing Pepe Silvia accounting for all the edge cases :)
I did enjoy this article and agree with almost everything.
One piece of advice I’ve learned the hard way, however, is if the road map doesn’t make sense, it’s unlikely that making noise is going to make your life better.
I was reminded by this article of the time when my role as a PM was stymied by a lack of clear corporate strategy - I was told to make a road map without the support of any kind of goal or vision, told even that having a strategy would take “11 weeks” (yeah, an oddly specific number). Of course, I pushed back - but the outcome of doing so was very poor for me.
A road map doesn’t need to perfectly align with the very good points in your article, but there is a point at which it’s probably just better to cut and run.
One piece of advice I’ve learned the hard way, however, is if the road map doesn’t make sense, it’s unlikely that making noise is going to make your life better.
I was reminded by this article of the time when my role as a PM was stymied by a lack of clear corporate strategy - I was told to make a road map without the support of any kind of goal or vision, told even that having a strategy would take “11 weeks” (yeah, an oddly specific number). Of course, I pushed back - but the outcome of doing so was very poor for me.
A road map doesn’t need to perfectly align with the very good points in your article, but there is a point at which it’s probably just better to cut and run.
It's so very rarely worth trying to fix pathological organizations.
[deleted]
> A great roadmap is all signal and little noise, so much so that the team can easily understand it and get started today.
That's in the realm of dreams, unless you're starting totally fresh, and have no dependencies at all.
That's in the realm of dreams, unless you're starting totally fresh, and have no dependencies at all.
Moreover, most roadmaps I've been a part of have all involved items where the first step is discovery, both product and technical. There's usually strong signal that it's the right thing to do, but the first step isn't writing code, it's figuring the thing out from different angles.
Exactly this.
The roadmap is not for the engineers to know what they are going to code next.
That comes through discussion and sprint planning in a tool that has tickets/milestones/epics etc.
The roadmap is not for the engineers to know what they are going to code next.
That comes through discussion and sprint planning in a tool that has tickets/milestones/epics etc.
[deleted]
Okay, engineers are now supposed to evaluate the roadmap to strategic goals. in principle all good that everybody is involved in lining up with strategy. But the caveat here is, the CPO and leadership team should do this. And if this does not resonate with the engineers, is it really wise to educate the product manager by sending him/her some literature on how to do strategy?
I would suggest to handle this if this occurs with more care. And also consider that many very successful companies did not have a written mission and strategy, it is usually introduced in the scale up phase when stakeholders demand more on the why and how.