AI Product Management(deeplearning.ai)
deeplearning.ai
AI Product Management
https://www.deeplearning.ai/the-batch/issue-279/
44 comments
How to use AI to write articles about how to use AI as a product manager for your AI app on your journey to being replaced by an AI product manager
How to rank at the top of HN using AI to write articles about how to use AI as a product manager for your AI app on your journey to being replaced by an AI product manager
yodawg.ai
Andrew Ng made another point about AI product management in a previous piece [1] that I found both thought-provoking and a bit contrarian, and I’m surprised he didn’t mention it here. In that earlier piece, he went beyond just advocating for concrete specs and explicitly challenged the traditional design-thinking approach, arguing that teams should pick a fully formed idea and run with it rather than spending too long on broad problem exploration and multiple potential solutions. It’s a stance that favors speed and specificity over the more open-ended, iterative nature of design thinking.
Curious what others think about forgoing design thinking in AI product development in favor of this more direct, concrete approach.
[1] https://www.deeplearning.ai/the-batch/concrete-ideas-make-st...
Curious what others think about forgoing design thinking in AI product development in favor of this more direct, concrete approach.
[1] https://www.deeplearning.ai/the-batch/concrete-ideas-make-st...
Advocating for waterfall?
Not every product can be totally designed and spec’d out from the outset. Especially when time to market is important.
Maybe this works at the individual feature scale, but at any reasonably large product, designing _everything_ from the outset would result in brittle design.
Not every product can be totally designed and spec’d out from the outset. Especially when time to market is important.
Maybe this works at the individual feature scale, but at any reasonably large product, designing _everything_ from the outset would result in brittle design.
Andrew Ng isn't proposing anything more than starting with a product idea rather than a market definition. "I've got an idea for a product so let's build a prototype to test it" isn't waterfall.
His argument is pointlessly contrarian, too. He says his proposal is counter to design thinking, but design thinking would encourage you to build the exact same prototype he is proposing. As his own piece acknowledges, if you're at an early stage where you don't have any specific product ideas, design thinking could be a good starting point.
In practice, this is all the same core idea. The end result is better if you investigate real ideas rather than rely on abstractions and assumptions. Test your ideas with prototypes. Be ready to discover your favourite idea doesn't work and change direction.
I wonder if he's really arguing against something that is independent of the method chosen: handing your money and control over to teams whose incentives are to spend as much time as possible on consultative exercises.
His argument is pointlessly contrarian, too. He says his proposal is counter to design thinking, but design thinking would encourage you to build the exact same prototype he is proposing. As his own piece acknowledges, if you're at an early stage where you don't have any specific product ideas, design thinking could be a good starting point.
In practice, this is all the same core idea. The end result is better if you investigate real ideas rather than rely on abstractions and assumptions. Test your ideas with prototypes. Be ready to discover your favourite idea doesn't work and change direction.
I wonder if he's really arguing against something that is independent of the method chosen: handing your money and control over to teams whose incentives are to spend as much time as possible on consultative exercises.
The comment I was replying to has this quote in it
> he went beyond just advocating for concrete specs and explicitly challenged the traditional design-thinking approach, arguing that teams should pick a fully formed idea and run with it rather than spending too long on broad problem exploration and multiple potential solutions
> he went beyond just advocating for concrete specs and explicitly challenged the traditional design-thinking approach, arguing that teams should pick a fully formed idea and run with it rather than spending too long on broad problem exploration and multiple potential solutions
> Not every product can be totally designed and spec’d out from the outset
I'd argue that no product can be spec'd 100% from the outset. Not even something like the regular Notepad.exe.
You'll always find some hidden complexity overlooked that results in the revision of the spec at the middle of development.
Embrace the change.
I'd argue that no product can be spec'd 100% from the outset. Not even something like the regular Notepad.exe.
You'll always find some hidden complexity overlooked that results in the revision of the spec at the middle of development.
Embrace the change.
There is common misconception among engineers about agile.
Agile is for reducing bunch of implementation docs. Your heavily crafted implementation docs will be gabage since implementation can be changed easily. However, we still need to define requirements and specs so that we won't create things no one want.
Agile is for reducing bunch of implementation docs. Your heavily crafted implementation docs will be gabage since implementation can be changed easily. However, we still need to define requirements and specs so that we won't create things no one want.
> Your heavily crafted implementation docs will be gabage since implementation can be changed easily.
I've heard this before but I'm not convinced. I'm sure lots of product teams use Agile and still manage to have thorough, up to date docs. You just need to factor doc updates into your issues.
I've heard this before but I'm not convinced. I'm sure lots of product teams use Agile and still manage to have thorough, up to date docs. You just need to factor doc updates into your issues.
Yes. This is intuitively true. Solid specs can be built with high % of AI assist and can be executed fast. Vague specs and exploration would take forever compared to the prior. One can build 10 concrete ideas in a span of a single exploration
I'm not sure some of this is a good idea. It reads a bit like "These LLMs are great! We can get rid of those pesky engineers!". It reminds me of the xkcd[1] about how some problems are trivial and some are almost impossible and as a layman you don't know which is which. That's more true than ever will LLMs, everything is new and so very few people actually know what is easy and what is hard. When you say "You can go off and do this without engineers and use AI to help you" what you are actually saying is "You can go and be a bad engineer". That's fine, if you don't have any engineers, then that's probably the best option. But if you're sensible probably a 5 minute conversation with someone who knows what they're talking about is more useful. In my experience as an engineer, the people who do jobs related to my job and think they can do my job: are crap at my job, generally not that good at their own either, and very difficult to work with.
I think the more interesting question for the PM is how are you going to make a differentiated product in the market if everything you're planning to build is trivial? If it's not trivial, maybe talk to an engineer or two.
[1]:https://xkcd.com/1425/
I think the more interesting question for the PM is how are you going to make a differentiated product in the market if everything you're planning to build is trivial? If it's not trivial, maybe talk to an engineer or two.
[1]:https://xkcd.com/1425/
Well I think it’s as simple as P vs NP. This is the primary difficulty with AI and always will be. Solutions are easy to get AI to construct. The difficulty is in verifying those solutions. This can be seen in industry even now, evals are the hardest part of non-trivial AI systems
I agree, but I think this also forms part of the solution: try to always ask the AI to create things which are easy to verify.
This describes the need to iterate on the product idea, in collaboration with engineering, instead of slinging vague PRDs over the wall.
Table stakes for any product manager, not just AI related.
Table stakes for any product manager, not just AI related.
Whenever a “good enough” prototype can be created just from a prompt, it becomes possible to scrap it every time it’s not quite right and regenerate, instead of editing the code. Working this way, the prompt is the “source code” and the code a kind of compiled artifact.
Currently the generated prototype usually needs tweaks and that’s if it even works. But when it does work, it’s like the model is reading your mind.
In the future as models improve at coding, they will anticipate the tweaks that make sense, less of the prompt will need to be specified & there’ll be less polish work after you get the generated artifact, and you can work at an even higher level of abstraction and thought. Domain experts can create even bigger, cooler things without spending’s years getting software engineering skills.
Assemblers and compilers came along very early in our industry’s history. If you run the thought experiment that that’s where we are at with prompted software creation, it will be a wild and exciting future. More people creating more stuff means a tremendous amount of amazing creations to enjoy.
Currently the generated prototype usually needs tweaks and that’s if it even works. But when it does work, it’s like the model is reading your mind.
In the future as models improve at coding, they will anticipate the tweaks that make sense, less of the prompt will need to be specified & there’ll be less polish work after you get the generated artifact, and you can work at an even higher level of abstraction and thought. Domain experts can create even bigger, cooler things without spending’s years getting software engineering skills.
Assemblers and compilers came along very early in our industry’s history. If you run the thought experiment that that’s where we are at with prompted software creation, it will be a wild and exciting future. More people creating more stuff means a tremendous amount of amazing creations to enjoy.
Wasn't that the idea behind SQL, then Excel, then no-code tools, etc. The second abstraction makes a previously complicated process simple the scope of expectations goes up.
The three basic guidelines are:
* Specify the product as concretely as possible
* Use existing applications to test feasibility
* Get non-engineer user feedback on early prototypes
These all obviously apply to product management more generally, but Andrew gives some examples/ways in which they apply specifically to AI products. Still, I feel like they're talking more generally about complex/abstract software engineering rather than simply AI.
* Specify the product as concretely as possible
* Use existing applications to test feasibility
* Get non-engineer user feedback on early prototypes
These all obviously apply to product management more generally, but Andrew gives some examples/ways in which they apply specifically to AI products. Still, I feel like they're talking more generally about complex/abstract software engineering rather than simply AI.
> Specify the product as concretely as possible
This is no small task.
This is no small task.
Indeed. I've seen people blind to how vague their specs were — I'm sure we all have a similar story, mine was someone wanting to know how much it would cost to make "uber for airlines" but it really was that vague a description even after I questioned them for more specifics, because their attempt at specifics was still vague.
So I guess the effect of the design decisions at the beginning will go towards zero as one approaches to the limits of the context window :)
The only thing that article says is: With AI you can iterate more rapidly/make prototypes faster.
Nothing new - we heard the same message with Figma, containerisation… you name it.
Having a good sense what problem solve, building rapport and trust with early customer and being a fantastic leader and communicatore has always been the most important skills. Thanks, nothing to see here…
Nothing new - we heard the same message with Figma, containerisation… you name it.
Having a good sense what problem solve, building rapport and trust with early customer and being a fantastic leader and communicatore has always been the most important skills. Thanks, nothing to see here…
You came away with a completely different understanding to the article than I did! I took it to be
"here's some guidelines for being a PM of an AI dev team"
Rather than
"Here's how to use AI to be a better PM"
"here's some guidelines for being a PM of an AI dev team"
Rather than
"Here's how to use AI to be a better PM"
I often bang on about Software as a new form of literacy - and that just as organisations that were staffed by literate people (Catholic churches, nation states, banks) were qualitatively different to organisations without literacy (stonemasons?) then software literate companies (I call them programmable companies in my upcoming book) will be different from the ones we know today.
And this is an example I feel of the form evolving.
The point that AI could not learn from a vague mission statement (whereas most people today would think wow that’s a good start to a two year project) suggests that AI companies as Ng suggests are “just” well thought out companies.
Sorry not making a lot of sense - what I think I mean is that one can write down a human sentence and the phase space of possible meanings is very large - the behaviours that meet the specification can be huge and most projects are attempts to find a working output that meets that and has everyone understanding it.
And this is an example I feel of the form evolving.
The point that AI could not learn from a vague mission statement (whereas most people today would think wow that’s a good start to a two year project) suggests that AI companies as Ng suggests are “just” well thought out companies.
Sorry not making a lot of sense - what I think I mean is that one can write down a human sentence and the phase space of possible meanings is very large - the behaviours that meet the specification can be huge and most projects are attempts to find a working output that meets that and has everyone understanding it.
But a *working* piece of software has a much more constrained phase space of possible behaviours - just to get it working (or even get a set of tests it must pass) drastically reduces the possible behaviours and so makes clearer intentions and makes the discussion more focused.Yeah but product management work has to happen prior to implementing the product. It sounds like you're saying you should implement the software first, then do product management on it.
it really is amazing how many people have no interest in doing a good job, nor even any interest in protecting their good name.
Easy to say if you are doing a worthwhile and interesting job in a good environment.
A lot of people (most?) do not have this. People who are overworked, people whose management want them to do things as cheaply as possible, people in physically or mentally bad environments.....
A lot of people (most?) do not have this. People who are overworked, people whose management want them to do things as cheaply as possible, people in physically or mentally bad environments.....
It really is amazing how many companies have no interest in upskilling their workers, nor even any interest in keeping long time experienced workers happy.
I think one can protect their good name without embracing the capitalistic grind.
For most people jobs are just a necessity to earn income to eat and have shelter.
Life for many is only counting outside office hours
To use AI can help me!That's best tools
Cron: bug devs about tickets that are late
Cron: bug devs every day at a given time for updates aka a standup
Given an epic with keywords organize tasks into that epic and estimate the time and then track if it’s on track or not.
Yeah not a lot to PM work.
Ooh also a 50/50 coin flipper to saying no to adhoc things
There that’s an AI PM
Given an epic with keywords organize tasks into that epic and estimate the time and then track if it’s on track or not.
Yeah not a lot to PM work.
Ooh also a 50/50 coin flipper to saying no to adhoc things
There that’s an AI PM
That's project management, not product management.
I’ll admit a good PM does interface well with stakeholders and gets them to finally agree to what they really need and to flesh out the requirements.
Otherwise it’s a role not too many teams need but when you need one you know it and you really need one.
Otherwise it’s a role not too many teams need but when you need one you know it and you really need one.
100%
A couple of cron jobs can easily automate this for my team. Most of the PMs I've seen in the wild only do this.
Very few PMs are actually valuable to a team from the product perspective.
A couple of cron jobs can easily automate this for my team. Most of the PMs I've seen in the wild only do this.
Very few PMs are actually valuable to a team from the product perspective.
Is it me or does this sound like a Project Manager and not a Product Manager? Where is the discussion and negotiation with customers and engineering teams on specs/targets/requirements? Where is the roadmap alignment with strategy and marketing teams? Where is the discussion with research teams on future features? Product managers don't babysit engineering teams on their deliverables. Calling someone doing the work listed on the grandparents post a product manager does not make them one.
>Calling someone doing the work listed on the grandparents post a product manager does not make them one.
Despite this, so many individuals stick around and try to make scrum their entire personality. It's insufferable at times to be around folks who do not even want to talk about customers or the product specifics.
Despite this, so many individuals stick around and try to make scrum their entire personality. It's insufferable at times to be around folks who do not even want to talk about customers or the product specifics.
Do AI customers next!