Breaking down tasks(jacobian.org)
jacobian.org
Breaking down tasks
https://jacobian.org/2024/mar/11/breaking-down-tasks/
117 comments
Holy... It still amuses me how much HN comments brings the reality to these kind of articles.
> the plan is never followed.
Sometimes it's even worst. As we add more tasks during execution, at the end, these tasks get twisted with the ones you created while planning. Then we end up with a messy list of undone tasks that aren't really required anymore (because they were created without having the full context, that only happens during the execution).
> the plan is never followed.
Sometimes it's even worst. As we add more tasks during execution, at the end, these tasks get twisted with the ones you created while planning. Then we end up with a messy list of undone tasks that aren't really required anymore (because they were created without having the full context, that only happens during the execution).
This is an accurate and insightful way of looking at work. It touches on something that I've been considering a lot lately: I love programming, but I hate doing it in a work environment. The fun thing about programming is that it is a flexible, flowing, and creative endeavor. You make things up as you go; it is an organic experience. In a work environment, this organic nature is removed mainly because there is a need for oversight and accountability.
I also can't get myself to do it seriously for my own tasks, but when I had to do it for a jr colleague I found out... yeah, breaking it down its hard, as hard as coding - then you get most of the hard work already done, and coding the solution becomes trivial. Chances of false starts go from 80% to 20%.
(A few times there is prototyping involved but I won't share that with non-devs, since they don't seem to get it.)
(A few times there is prototyping involved but I won't share that with non-devs, since they don't seem to get it.)
I am somewhat like that myself. I do formulate a plan and break down tasks in advance, but I give myself an absolute right to change and modify it at will whenever new ideas come to light or just because I'm feeling like it.
> the plan is never followed
The plan not being perfect and prescient is part of the plan.
Next time you plan for same or similar activity, your future plan will be better.
The Project Managers even have a mantra: "fail to plan is plan to fail."
The plan not being perfect and prescient is part of the plan.
Next time you plan for same or similar activity, your future plan will be better.
The Project Managers even have a mantra: "fail to plan is plan to fail."
And Eisenhower's "plans are useless, planning is essential", of course. I know.
But for me it's very easy to overdo it. Give me interesting chunks of about a week or two, with a tight deadline, and I'll do my best work.
Remember, Linus wrote Git in two weeks. And there was no project manager in sight.
But for me it's very easy to overdo it. Give me interesting chunks of about a week or two, with a tight deadline, and I'll do my best work.
Remember, Linus wrote Git in two weeks. And there was no project manager in sight.
On the Linus point, I'd bet there were countless months where he was bringing ideas together in his head and planning how he'd do it before actually finding the time to sit down and write it.
It would be interesting to know whether he had devised some kind of project plan ahead of time. But I suspect not. I'm sure he had been thinking about the technical details of what he wanted to build - which yep, we all like to do that! - but I doubt he bothered to break down a set of approximately-ordered 1- to 3-day work units, which is what many of us struggle to be good at.
[deleted]
Whenever you talk about breaking down tasks, estimations etc, those usually are for multi-person teams, and/or projects that have some specific budget or something.
Of course if you are exploring or building your own projects and/or don't have any other accountability, you wouldn't start planning that.. unless you are really into planning in and of itself.
But as soon as you have a boss and the boss asks "how long? who does what? where do we start?" - you have to have some framework.
I also have a very high aversion to "conforming to the system", esp. when these systems seem arbitrary or are overly rigid .. but systems in general need to be simple and flexible. Productivity systems are there to help people.
I don't think the author intended for this to be like a detailed recipe.. but more like an example of how he approaches these things, which can hopefully help others have their own ideas.
Of course if you are exploring or building your own projects and/or don't have any other accountability, you wouldn't start planning that.. unless you are really into planning in and of itself.
But as soon as you have a boss and the boss asks "how long? who does what? where do we start?" - you have to have some framework.
I also have a very high aversion to "conforming to the system", esp. when these systems seem arbitrary or are overly rigid .. but systems in general need to be simple and flexible. Productivity systems are there to help people.
I don't think the author intended for this to be like a detailed recipe.. but more like an example of how he approaches these things, which can hopefully help others have their own ideas.
> But as soon as you have a boss and the boss asks "how long? who does what? where do we start?" - you have to have some framework.
Why?
Does it work? Is it productive? Is it better than not doing that?
Why?
Does it work? Is it productive? Is it better than not doing that?
Suppose it doesn’t work, it’s not productive, and it’s worse than not doing it.
You still have a boss asking these questions. Any answer you provide constitutes a framework.
You still have a boss asking these questions. Any answer you provide constitutes a framework.
Well, one answer you can try is to push back on the concept of providing status updates. For many (but not most) high-trust individuals working on many (but I think not most) kinds of projects, that's totally fine. (Ideally it is an expectation set much earlier on than the point where a status update is being requested, though.)
On average: yes, yes, and yes.
> Does it work? Is it productive? Is it better than not doing that?
>> On average: yes, yes, and yes.
For what it's worth, I think it's more like: On average: no, no, and yes.
It doesn't really "work" in the sense that the plan turns out to be an accurate and useful reflection of the work; that is not the case on average in my experience (sometimes it is, but not usually). And for the same reason, it isn't usually a productive exercise; it usually costs more time up front than it earns in productivity.
But despite those two things I think it still turns out to be better on average, in a team / "you have a boss" environment. Because productivity is not the only thing that matters, legibility is also important to organizations, and worth some productivity cost. Though this will chafe me until I'm in my grave, I still think it's true.
>> On average: yes, yes, and yes.
For what it's worth, I think it's more like: On average: no, no, and yes.
It doesn't really "work" in the sense that the plan turns out to be an accurate and useful reflection of the work; that is not the case on average in my experience (sometimes it is, but not usually). And for the same reason, it isn't usually a productive exercise; it usually costs more time up front than it earns in productivity.
But despite those two things I think it still turns out to be better on average, in a team / "you have a boss" environment. Because productivity is not the only thing that matters, legibility is also important to organizations, and worth some productivity cost. Though this will chafe me until I'm in my grave, I still think it's true.
The plan being an accurate reflection of what then happens is not the point.
Having people assigned to tasks, a starting point, some initial steps to follow is something that increase delivery speed in the beginning a lot.
Edit: to some extent, legibility is productivity, as it means others are aware of what is going on and can interact/refer/etc.
Having people assigned to tasks, a starting point, some initial steps to follow is something that increase delivery speed in the beginning a lot.
Edit: to some extent, legibility is productivity, as it means others are aware of what is going on and can interact/refer/etc.
> Having people assigned to tasks, a starting point, some initial steps to follow is something that increase delivery speed in the beginning a lot.
It's a balance of time spent planning against the value of the plan. Immediate-term "planning" is definitely valuable, IMO. Thinking through how some immediate work can be split up so that multiple people can make progress simultaneously, without stepping on each other's toes, that is definitely valuable. Having some discussion about what the next steps might be after those immediate steps are complete, that's also pretty much always valuable. Spending a small amount of time thinking about the next next steps can be valuable. But past that, in my experience, people often put more time into planning detailed steps than ends up being valuable. It's not that it has zero value, it's that it has a pretty high cost that, in my opinion, usually outweighs that value.
Also, just to note that I think spending time thinking about a "north star" vision of what a project is trying to achieve is also pretty much always strongly positive ROI.
It's time spent coming up with a detailed plan a few steps out from the immediate term that I find dubious. I don't mind other people spending that time if they want to, but I don't like spending my time in those planning meetings; I'd rather be getting to work on the immediate term tasks.
It's a balance of time spent planning against the value of the plan. Immediate-term "planning" is definitely valuable, IMO. Thinking through how some immediate work can be split up so that multiple people can make progress simultaneously, without stepping on each other's toes, that is definitely valuable. Having some discussion about what the next steps might be after those immediate steps are complete, that's also pretty much always valuable. Spending a small amount of time thinking about the next next steps can be valuable. But past that, in my experience, people often put more time into planning detailed steps than ends up being valuable. It's not that it has zero value, it's that it has a pretty high cost that, in my opinion, usually outweighs that value.
Also, just to note that I think spending time thinking about a "north star" vision of what a project is trying to achieve is also pretty much always strongly positive ROI.
It's time spent coming up with a detailed plan a few steps out from the immediate term that I find dubious. I don't mind other people spending that time if they want to, but I don't like spending my time in those planning meetings; I'd rather be getting to work on the immediate term tasks.
I think making plans/planning needs training to be good at - which in turn reduces "overplanning". Knowing what to include, discuss in which forum, what task or deliverable granularity, etc. is useful doesn't come naturally to most.
Agreed.
But actually, I kind of feel like people who have had training to get good at planning, are not the most likely people to be good at avoiding "overplanning". I think it's the same kind of incentives as how dentists have a natural bias toward recommending dental work, or chiropractors toward recommending adjustments, or programmers toward recommending writing software, or anyone toward doing a bit more of whatever it is they are trained at and good at than a totally unbiased third party might really need or want.
Maybe there's also a bit of a "midwit meme" to this (in all of these cases): inexperienced: don't do it!, some experience (midwit): do it!, extremely experienced: don't do it ... except in these cases where you really should, and include just the right amount, and be very thoughtful about the forum and who to include in the discussion, and calibrate the right level of granularity for this specific project, and ...
But actually, I kind of feel like people who have had training to get good at planning, are not the most likely people to be good at avoiding "overplanning". I think it's the same kind of incentives as how dentists have a natural bias toward recommending dental work, or chiropractors toward recommending adjustments, or programmers toward recommending writing software, or anyone toward doing a bit more of whatever it is they are trained at and good at than a totally unbiased third party might really need or want.
Maybe there's also a bit of a "midwit meme" to this (in all of these cases): inexperienced: don't do it!, some experience (midwit): do it!, extremely experienced: don't do it ... except in these cases where you really should, and include just the right amount, and be very thoughtful about the forum and who to include in the discussion, and calibrate the right level of granularity for this specific project, and ...
I think that happens typically when the planning and the execution are very divorced (someone good at/selling using the techniques of planning). I'd say a good operational planer doesn't overplan.
I agree, but I guess the point I was trying to make is that I think everybody is susceptible to over-biasing toward thinking the thing they know how to do is super important, to the point that they sometimes over-do it, despite their best intentions.
I would say that "a good programmer doesn't write more software than is necessary", but despite my best intentions, I'm certain that I write more software than is necessary, because writing software is my hammer and lots of things look like nails to me.
I think it requires more than just being "good" at something to overcome this tendency. I'm sure the very best operational planners, like the very best software engineers, consistently strike the perfect balance, but I suspect most "good" planners, like most "good" (even "great") software engineers I know, still struggle with this tendency.
I would say that "a good programmer doesn't write more software than is necessary", but despite my best intentions, I'm certain that I write more software than is necessary, because writing software is my hammer and lots of things look like nails to me.
I think it requires more than just being "good" at something to overcome this tendency. I'm sure the very best operational planners, like the very best software engineers, consistently strike the perfect balance, but I suspect most "good" planners, like most "good" (even "great") software engineers I know, still struggle with this tendency.
> I never ever end up doing the actual steps as I planned to
I do the GTD thing of only thinking about the _next_ action, not trying to think of every action.
I do the GTD thing of only thinking about the _next_ action, not trying to think of every action.
Seriously asking : how? I can't cheat my brain into not automatically seeing the next steps after that and the cascading relation between the all. Within a second or less.
A related if not the same thing I've been trying to do, especially with some DIY type work, is that it doesn't matter, deal with that later, just get this done thing for now.
I.e. not somehow trick yourself into not thinking about it, but just think who knows if/when I'll actually get around to that, for now just do this. Don't worry about optimal order.
If the skirting doesn't look as great as it could because I fixed and painted the doorframe first, only painting the wall and rest of skirting later, meh who cares, if it turns out to be that noticeable maybe I'll get around to giving it all another coat; in the meantime at least I got it done, not stuck in 'analysis paralysis'.
I.e. not somehow trick yourself into not thinking about it, but just think who knows if/when I'll actually get around to that, for now just do this. Don't worry about optimal order.
If the skirting doesn't look as great as it could because I fixed and painted the doorframe first, only painting the wall and rest of skirting later, meh who cares, if it turns out to be that noticeable maybe I'll get around to giving it all another coat; in the meantime at least I got it done, not stuck in 'analysis paralysis'.
Tangent: I can't recall where I first heard "plans are worthless, but planning is essential", but there's definitely some truth in it.
Another perspective is that planning is a breadth-first traversal of the solution space, and coming up with a path to the solution. When reality hits and the path is often wrong, one can switch to other paths quickly since the graph was created ahead of time. It's writing the table of contents for a book before fleshing it out.
Without planning, a depth first traversal is a high risk endeavor in the likelihood that the that path is wrong but backtracking and creating the graph is comparatively expensive and susceptible to sunk cost fallacy. Depth-first traversal is writing the book a chapter at a time without a table of contents in mind.
Without planning, a depth first traversal is a high risk endeavor in the likelihood that the that path is wrong but backtracking and creating the graph is comparatively expensive and susceptible to sunk cost fallacy. Depth-first traversal is writing the book a chapter at a time without a table of contents in mind.
Certainly holds true for having a baby.
You can plan all you want and it is essential... however you'll quickly find out that most of your plans are in your newborns' hands.
You can plan all you want and it is essential... however you'll quickly find out that most of your plans are in your newborns' hands.
Yes! Also Mike Tyson's famous "Everybody has a plan, until they get punched in the face."
For babies, everybody has a plan until they get pooped on / thrown up on.
Dwight Eisenhower, who also created The Matrix.
For clarification, "The Matrix" refers to the urgency vs importance decision matrix and not the movie: https://asana.com/resources/eisenhower-matrix
It's a framework to prioritize important tasks instead of falling into the agency trap, akin to prioritizing meaningful strategic tasks such as product development and tech debt instead of fighting fires.
It's a framework to prioritize important tasks instead of falling into the agency trap, akin to prioritizing meaningful strategic tasks such as product development and tech debt instead of fighting fires.
I'm a person who hates lists and plans for personal tasks. But, am learning to love them. Because as I often realize, things get missed in the myriad of things to remember.
The plan to me is just a way of reminding my future self what I thought of as ideal output in the past. Instead of infinitely changing my plans and chasing fireflies
The plan to me is just a way of reminding my future self what I thought of as ideal output in the past. Instead of infinitely changing my plans and chasing fireflies
That's slightly different, IMO. Writing down things that need to get done as you think of them is not the same as writing down a plan.
I make lists and break tasks in to smaller tasks, however I rarely reference these lists. They basically end up in the void. The act of making the list gets my juices flowing and reminds me of the side effects / bigger picture of my current challenge.
Realizing this also helps me avoid yak shaving over list making tools, since I could care less about even saving the lists! I keep a folder of text files in any project I'm working to hold my lists, but tbh it's unnecessary to even save them.
Realizing this also helps me avoid yak shaving over list making tools, since I could care less about even saving the lists! I keep a folder of text files in any project I'm working to hold my lists, but tbh it's unnecessary to even save them.
Sometimes planning is more important than having a plan
I see this a lot in this thread (and have heard it a number of times before), and it sounds intuitively right to me, but could you give an example of when and why that is the case?
Writing an idea down just seems to reinforce it and makes you flesh it out. I think it engages your brain differently, but I’m no neuroscientist.
Even if this is right (and I definitely think there is a lot of truth to it!), planning is not the only kind of "writing down ideas to reinforce and flesh them out" activity. Is it the most valuable one? And is this the only and best way to reinforce and flesh out ideas, or are there other ways that might be even better, like creating prototypes or just actively fiddling around with different approaches and possibilities? In my view, it isn't that there is no value in pondering the detailed sequence of work it will require to accomplish something over the next three months, it's more that I think beyond a horizon of a few weeks, most of the other things I could be doing with that time usually ends up seeming more valuable, to me.
> Delivers change because, in a work context, a task only “matters” if something is different because of the work.
I think maintenance tasks may require more effort / more expansive qualification when it comes to the meaning(s) of "something is different".
I think the topic of "breaking down tasks" could very well be its own book genre or even podcast genre. My experience with self-help and organization titles has been that the activity of breaking down tasks is a sort of known primitive human operation that many authors assume their readers to have. However, my own personal experience and the accounts of others I have heard while participating in group sessions is that breaking down tasks can be very difficult and provoke emotions of avoidance and despair.
The most broadly relevant advice that I have come across when it comes to breaking down tasks is to keep breaking things down until I am 90% certain that I can successfully complete the task. The certainty here would vary on how much self-confidence an individual has and their risk tolerance, so some people may break down tasks to 70% certainty of success.
I think maintenance tasks may require more effort / more expansive qualification when it comes to the meaning(s) of "something is different".
I think the topic of "breaking down tasks" could very well be its own book genre or even podcast genre. My experience with self-help and organization titles has been that the activity of breaking down tasks is a sort of known primitive human operation that many authors assume their readers to have. However, my own personal experience and the accounts of others I have heard while participating in group sessions is that breaking down tasks can be very difficult and provoke emotions of avoidance and despair.
The most broadly relevant advice that I have come across when it comes to breaking down tasks is to keep breaking things down until I am 90% certain that I can successfully complete the task. The certainty here would vary on how much self-confidence an individual has and their risk tolerance, so some people may break down tasks to 70% certainty of success.
The problem I have with task breakdowns is you get way too much engineer overconfidence - i.e. there is literally no task which is less then an entire day's effort, in practice. There's about 6 usable hours in a day - I've watched people confidently bid they'll get something done in "half a day" but when you point out that's about 3 hours, suddenly they're way less sure about that number - or offended. Then 3 days later they're still working on it.
Sometimes it is not overconfidence but a side-effect of management interference & second-guessing the estimates.
In this case engineers give an estimate of what they think management wants to hear and not the actual time it takes. It happens if they are frequently asked to reduce their estimated time to within management expectations.
To deal with this, you need to bargain with them in the opposite direction to make sure the estimates are realistic.
In this case engineers give an estimate of what they think management wants to hear and not the actual time it takes. It happens if they are frequently asked to reduce their estimated time to within management expectations.
To deal with this, you need to bargain with them in the opposite direction to make sure the estimates are realistic.
Managers are typically squeezed between the person who "wants the project done, and sets the budget" and the technical folk doing the work.
Kinds like the architect sitting between the client and the building contractor. The client wants the moon, on a low budget, and wants it completed yesterday.
A strong manager understands their role, and the constraints (time, money) of the system and is able to find compromise where necessary. They're there to guide the owner, and at the same time monitor the workers.
Of course the vast majority of middle management are not strong. They see their role as passing on orders from on high. The boss is the customer, and the customer is always right. So their only part is to demand more from the workers.
Strong tech workers understand what the manager needs. Good time estimates. Limited budget. The manager needs to be passing accurate data upstream. Bad workers ignore reality, tell a manager what he wants to hear, and just do their thing. Not realising that they (the programmers) will be the ones thrown under the bus ehen it goes tits up. (Another sign of bad managers is to blame the underlings.)
Good managers, working with good techies, is a match made in heaven. Together they deliver accurate estimates, with plenty of contingency time. Together they decide what features are necessary, and what can be canned. If you are in this dynamic, count yourself lucky.
If you are a good manager working with crap staff, well, that's unlucky. But at least you can turn them over. If you are a good tech working with a crap manager (which I suspect is most of us) then, well, I guess the only options are to stay, or quit.
Bad managers can be made better with explanations though. The more they understand the constraints, the better equipped they are to argue the point with the customer.
Kinds like the architect sitting between the client and the building contractor. The client wants the moon, on a low budget, and wants it completed yesterday.
A strong manager understands their role, and the constraints (time, money) of the system and is able to find compromise where necessary. They're there to guide the owner, and at the same time monitor the workers.
Of course the vast majority of middle management are not strong. They see their role as passing on orders from on high. The boss is the customer, and the customer is always right. So their only part is to demand more from the workers.
Strong tech workers understand what the manager needs. Good time estimates. Limited budget. The manager needs to be passing accurate data upstream. Bad workers ignore reality, tell a manager what he wants to hear, and just do their thing. Not realising that they (the programmers) will be the ones thrown under the bus ehen it goes tits up. (Another sign of bad managers is to blame the underlings.)
Good managers, working with good techies, is a match made in heaven. Together they deliver accurate estimates, with plenty of contingency time. Together they decide what features are necessary, and what can be canned. If you are in this dynamic, count yourself lucky.
If you are a good manager working with crap staff, well, that's unlucky. But at least you can turn them over. If you are a good tech working with a crap manager (which I suspect is most of us) then, well, I guess the only options are to stay, or quit.
Bad managers can be made better with explanations though. The more they understand the constraints, the better equipped they are to argue the point with the customer.
this, the number of times I've heard: I can't sleep those estimates to the customer. How is that a me problem? I don't care how you sell it, just know that my estimate is pretty accurate on how much time we need to build it.
And also don't expect us to work 8 hours a day doing tasks.
And also don't expect us to work 8 hours a day doing tasks.
Estimating time for a task is it's own skill. And, of course, the danger is in the task being insufficiently specified, leading to unknowns.
Cook lunch, estimate 30 minutes. Task starts, oh wait, cupboard is bare, need to go shopping.
Over a long career I've found that you do the best estimate you can with the information provided. Then multiply by 4. Sounds extreme, but you're still likely to be short a bit.
Equally, for estimates, I like to first estimate the number of data tables. That's a good factor to bring in to the rest. A system with 10 tables, 100 tables, 200 tables and so on "magnifies" the task list.
"Build reports" has a proportionate number. 10 tables might mean 5 reports. 100 tables might mean 50.
Building reports might be easy, but the number of them means time is needed.
Cook lunch, estimate 30 minutes. Task starts, oh wait, cupboard is bare, need to go shopping.
Over a long career I've found that you do the best estimate you can with the information provided. Then multiply by 4. Sounds extreme, but you're still likely to be short a bit.
Equally, for estimates, I like to first estimate the number of data tables. That's a good factor to bring in to the rest. A system with 10 tables, 100 tables, 200 tables and so on "magnifies" the task list.
"Build reports" has a proportionate number. 10 tables might mean 5 reports. 100 tables might mean 50.
Building reports might be easy, but the number of them means time is needed.
This is close to my rule - whatever I think, multiply by 3. Then multiply by 3 if it includes any amount of "get another team, under a different manager, to do something". It's been accurate more times then I care to think.
My current project estimate is 1 year. I'm afraid if I need to multiply that by 3.
> Over a long career I've found that you do the best estimate you can with the information provided. Then multiply by 4. Sounds extreme, but you're still likely to be short a bit.
Me myself rely on Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
Me myself rely on Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
But sometimes I see a task that would take about a day... and then people insist on breaking it down in four parts!
Then they take about a quarter day each, provided the same person does all four simultaneously. Or about a day each if you give them to different people.
Then they take about a quarter day each, provided the same person does all four simultaneously. Or about a day each if you give them to different people.
Similar: the more uncertainty, the further I break it down.
This has been amazingly accurate for time estimates - in aggregate... Individually, they are wildly inaccurate.
So... they are probability estimates. Over many events, they even out.
This has been amazingly accurate for time estimates - in aggregate... Individually, they are wildly inaccurate.
So... they are probability estimates. Over many events, they even out.
Yes, maintenance tasks consume more resources. After all, that's what the classic SDLC methodologies state.
Software development cannot be managed like this. This sort of task break downs come from classic management training. The problem most people are unaware of is software development is more a creative activity than anything else. Yes there are serious technical aspects to it but since the problem is virtual and not bound by any real world limitations, like how civil engineering would be, there isn't one optimal solution. Trying to define the solution before the problem has been properly examined would only limit the final output. Most of the exploration only happens when people actually start coding.
Not having a well defined final output and time is not very relevant for software, mostly because there is no per unit cost. Most managers do not realize this is they are not from a software engineering background. A versatile product can be sold to many different customers without additional development costs.
But since most companies are managed as factories, all the processors will ultimately create a very limited product targeted at a specific customer. What the big tech companies have avoided is this .
Not having a well defined final output and time is not very relevant for software, mostly because there is no per unit cost. Most managers do not realize this is they are not from a software engineering background. A versatile product can be sold to many different customers without additional development costs.
But since most companies are managed as factories, all the processors will ultimately create a very limited product targeted at a specific customer. What the big tech companies have avoided is this .
You'd be surprised how much creative tasks like 3D modeling and creating artwork can be broken down into tasks that are very well defined and can be estimated quite correctly.
I think it would fall down the same lines if you required a coherent world and high quality for each of the models you create.
That would mean redo models that don't fit the whole piece, endless tweak some models that are really difficult to get right, sometimes do swiping changing across all the parts because as a team you realize something is off etc.
The balance will be between delivering something within a time frame, and delivering the quality that was requested/expected.
That would mean redo models that don't fit the whole piece, endless tweak some models that are really difficult to get right, sometimes do swiping changing across all the parts because as a team you realize something is off etc.
The balance will be between delivering something within a time frame, and delivering the quality that was requested/expected.
The difference is that you break down software into smaller and smaller bits and then find a small bit that needs data from a far away small bit. I suspect this happens less with mechanical systems.
If that’s the case, what makes the work “creative”? (It’s possible two sets of people use the term to mean two different things)
> But since most companies are managed as factories, all the processors will ultimately create a very limited product targeted at a specific customer.
Do you have more examples of companies that are not feature factories? It seems the entire industry has adopted this pattern.
Do you have more examples of companies that are not feature factories? It seems the entire industry has adopted this pattern.
> Do you have more examples of companies that are not feature factories?
Actually, none of them are. They're just managed as if they were.
In a factory, you can literally have a machine that stamps out 1000 widget blanks an hour and if the operator does an hour of overtime, that's an extra 1000 widgets produced that day.
You've got your workers on 8-hour shifts and sales signs a contract to deliver 8,500 widgets per day? Then you know right away you're going to need 10-hour shifts, or regular overtime, or a second 8-hour shift, or a second machine, and you can figure out how much each of those would cost, and whether you've got enough car parking for the extra workers.
Whereas in software, your feature factory produces ???? per hour, sales promises customers ???? and an hour of overtime achieves ???.
Actually, none of them are. They're just managed as if they were.
In a factory, you can literally have a machine that stamps out 1000 widget blanks an hour and if the operator does an hour of overtime, that's an extra 1000 widgets produced that day.
You've got your workers on 8-hour shifts and sales signs a contract to deliver 8,500 widgets per day? Then you know right away you're going to need 10-hour shifts, or regular overtime, or a second 8-hour shift, or a second machine, and you can figure out how much each of those would cost, and whether you've got enough car parking for the extra workers.
Whereas in software, your feature factory produces ???? per hour, sales promises customers ???? and an hour of overtime achieves ???.
Bell Labs?
Changing the topic from software development to R&D: the granular task oriented view is generally not good for greenfield research endeavours. It's a form of premature optimization. You're calcifying the search space and reducing flexibility to pivot immediately based on expert intuition and hunches. That being said, a coarse task orientation is necessary to keep people vaguely pointed in the right direction. The best resolution (time horizon) for task orientation depends on what you're doing.
Changing the topic from software development to R&D: the granular task oriented view is generally not good for greenfield research endeavours. It's a form of premature optimization. You're calcifying the search space and reducing flexibility to pivot immediately based on expert intuition and hunches. That being said, a coarse task orientation is necessary to keep people vaguely pointed in the right direction. The best resolution (time horizon) for task orientation depends on what you're doing.
Do you have any pointers for guides about how to implement this sort of methodology in software-based R&D? I'm currently in a small, partly research-focused team as part of a larger tech company, and it's proving a struggle to align our work with the rest of the dev squads who do all their work in 2-week sprints with max-2-day tasks. I'd love to better understand how research teams tackle this problem in other companies. It would be amazing if I could present an alternative strategy to management that's worked elsewhere.
The research breakthroughs will come from self-motivated, highly competent people. Figure out who they are, give them resources/help and get out of their way. Let them do nothing for 48 hours because it isn't nothing, they're thinking about what should be done. You're asking them to navigate a tremendous search space, and that will require unconventional modes of working. If you hired the right people, they should know better than you about how to best navigate that space. Keep them accountable over a 3-6 month time horizon, not a 1 week horizon. If they're juniors, they will need daily or weekly guidance, but not dictates or deadlines. Deadlines for artificial milestones do not make for better results, they degrade results because they force time-consuming proof of work. Researchers sometimes need active prodding if they are stuck in a local optima, but that's a different beast to predetermining specific tasks a week in advance. I'm assuming the talent bar is kept high so they can autonomously work in the above fashion, and I am assuming they are intrinsically motivated to produce results.
When it comes to less intrinsically motivated team members, I don't have useful advice. Maybe they can contribute to research output, but that would require more traditional management styles which I can't comment on.
When it comes to satisfying constraints imposed by other parts of the org, I can't usefully comment on this either.
When it comes to less intrinsically motivated team members, I don't have useful advice. Maybe they can contribute to research output, but that would require more traditional management styles which I can't comment on.
When it comes to satisfying constraints imposed by other parts of the org, I can't usefully comment on this either.
Thanks. It is the "satisfying other parts of the org" part I'm struggling with, but it's helpful to see the reasoning laid out as you do in your first paragraph.
I guess the essential problem I'm trying to solve is how to respond when the MD or a product owner comes to me and says "The client wants our core algorithms to be adapted in XYZ novel ways. When can you deliver this?".
I guess the essential problem I'm trying to solve is how to respond when the MD or a product owner comes to me and says "The client wants our core algorithms to be adapted in XYZ novel ways. When can you deliver this?".
Interesting you mention 'intuition'. I feel it is a big part of software development as you gain experience and it flows through as you are doing the work and would be tough to plan for. Also remember one of my colleagues complaining after work 'code just not flowing freely today'. And then there are times when you feel you are in the zone, which would definitely be the creative element.
> Bell Labs?
Too bad that's pretty much the only such example that has been touted since... The 1960s, is it? And that it was, IIRC, recently either shut down or announced to be shut down sometime soon.
Thus, using this as an example seems, if anything, to illustrate not that "there are too!" but "there are dwindingly few".
Too bad that's pretty much the only such example that has been touted since... The 1960s, is it? And that it was, IIRC, recently either shut down or announced to be shut down sometime soon.
Thus, using this as an example seems, if anything, to illustrate not that "there are too!" but "there are dwindingly few".
Being a career long engineer, I'm not unaccustomed to breaking down huge projects into smaller things that can be parallelized, plotted in time, etc. We have to do it and get better at it.
But honestly, I think what holds most of us back is a better ability to just don't do that. Take that thing you want to make, and instead of planning out all the pieces, just make the tiniest possible thing that could possibly be of value. Using this example, just start with "Today" and the 4 buttons. Make a thing that shows your 4 buttons of exercise you will click on each day. You can probably make that today. No streaks or freezes or calendar view. Those are all great ideas, and you'll get to them. But you need momentum. And you'll probably decide you don't even like that calendar view in a few days. I think more projects need a kick in the pants of just do something small. Get it done today. See what the next task is from that point because it's probably not what you thought it ought to be during the planning phase.
This also isn't exactly easy either. It requires some thought and discipline and find the smallest thing and commit to shipping it without distraction. But it's also something good to practice at.
But honestly, I think what holds most of us back is a better ability to just don't do that. Take that thing you want to make, and instead of planning out all the pieces, just make the tiniest possible thing that could possibly be of value. Using this example, just start with "Today" and the 4 buttons. Make a thing that shows your 4 buttons of exercise you will click on each day. You can probably make that today. No streaks or freezes or calendar view. Those are all great ideas, and you'll get to them. But you need momentum. And you'll probably decide you don't even like that calendar view in a few days. I think more projects need a kick in the pants of just do something small. Get it done today. See what the next task is from that point because it's probably not what you thought it ought to be during the planning phase.
This also isn't exactly easy either. It requires some thought and discipline and find the smallest thing and commit to shipping it without distraction. But it's also something good to practice at.
I have come to think of this as "The Anna Principle" in organizing my own work:
> when faced with uncertainty, one must simply focus on doing "The Next Right Thing." [0]
This is not to trivialize it! Figuring out the next right thing to do is hard, but also the most valuable part of planning, in my view.
0: https://en.wikipedia.org/wiki/The_Next_Right_Thing#Synopsis
> when faced with uncertainty, one must simply focus on doing "The Next Right Thing." [0]
This is not to trivialize it! Figuring out the next right thing to do is hard, but also the most valuable part of planning, in my view.
0: https://en.wikipedia.org/wiki/The_Next_Right_Thing#Synopsis
OMG, I love that you put Frozen 2 and princess Anna and a song onto this. Thank you for this coinage. I will absolutely be using this now. Please write a blog post on your coinage of this so that we all here can link to that and give you some credit. I'm serious. I looked up The Anna Principle and I wanted to see your blog on this.
:) Alas, I do not blog.
"Do the next right thing" was a phrased used by Alcoholics Anonymous long before that 2019 movie came out, so I don't think calling it "The Anna Principle" is appropriate.
I didn't know this! That just makes me appreciate this more, and thank you for making me aware of it.
But thinking of it as "The Anna Principle", named after a character from a children's movie, remains way more fun. It was already a tongue-in-cheek-ism to ascribe what I really do believe is a strong foundational idea (not just in project planning, but in life) to a character in a children's movie.
By the way, I tracked this back to at least one earlier use of a similar construction (which maybe is implied to be the basis of its use in AA) by Carl Jung:
> But if you do with conviction the next and most necessary thing, you are always doing something meaningful and intended by fate.
Excerpted from: https://www.themarginalian.org/2021/12/07/carl-jung-next-rig...
But I suspect this is an even older, indeed timeless, idea. But ascribing it to a modern well-known movie character makes it more fun and memorable :)
But thinking of it as "The Anna Principle", named after a character from a children's movie, remains way more fun. It was already a tongue-in-cheek-ism to ascribe what I really do believe is a strong foundational idea (not just in project planning, but in life) to a character in a children's movie.
By the way, I tracked this back to at least one earlier use of a similar construction (which maybe is implied to be the basis of its use in AA) by Carl Jung:
> But if you do with conviction the next and most necessary thing, you are always doing something meaningful and intended by fate.
Excerpted from: https://www.themarginalian.org/2021/12/07/carl-jung-next-rig...
But I suspect this is an even older, indeed timeless, idea. But ascribing it to a modern well-known movie character makes it more fun and memorable :)
One thing that works for parallelization in these unsecure situations are things that you know you'll surely need. This even might just involve testing things. E.g. on member developes a comparison method for testing, another member goes for one approach, another for the other and a third for yet another. And after a set period you try to evaluate everything. Of course this approach only makes sense when there isn't already a favoured or clearly superior approach.
Similarily during actual development modularization can be a way to parallelize development. So one component per person or per team.
For my own stuff I also like to interleave the human-facing parts of my products with the technological guts, especially if I am still figuring things out — developing the technology without knowing how it is being used will force the way you can use it into a certain direction — designing the interface without knowing the technology has it's own traps as well.
Similarily during actual development modularization can be a way to parallelize development. So one component per person or per team.
For my own stuff I also like to interleave the human-facing parts of my products with the technological guts, especially if I am still figuring things out — developing the technology without knowing how it is being used will force the way you can use it into a certain direction — designing the interface without knowing the technology has it's own traps as well.
This is typically called a PoC or MVP - shouldn't this also be scheduled and forecasted?
Yes, definitely thought of that way most of the time. But I also think that might actually trivialize this concept some. Because one might say "ok, here's the crappy MVP version, and now that we're out of that one month phase everything after requires very rigorous and careful planning and meetings". I think most projects actually need to just stay in MVP mode. The entire project or product will be forever in a place of high uncertainty. Every feature should feel like it was the minimum we could do.
I'm actually not saying every project needs this of course, but I think maybe most might? Obviously there's huge things I've been apart of, and we needed to be thorough planning step by step months out. But that experience should probably be the exception and not the rule. Minimums the rule.
I'm actually not saying every project needs this of course, but I think maybe most might? Obviously there's huge things I've been apart of, and we needed to be thorough planning step by step months out. But that experience should probably be the exception and not the rule. Minimums the rule.
I find that this attitude is also great for life in general.
Absolutely. I mean, yes, there's those moments where you need to plan your retirement and you better think 40-50-60 years out. But most of the time, just do the next 5 minute thing and see what happens. I've been getting a lot better recently about just coming up with 3 "quests" I'm going to do today that probably take 5 minutes. And that little progress has kept building and building into something I didn't really see months ago when I started that first 5 minute step.
Breaking down work works great until the breakdown is unknowable. Things like research, that require creative experiments to POC things that are not known apriori. That’s when task breakdown breaks down.
...and then you break out the actuarial models :)
And at some point, you just gotta do it.
And at some point, you just gotta do it.
Also when management sees what should be a POC or research as a deliverable they start promising to higher ups and other teams. I’ve developed the habit of doing most of my POC work behind closed doors and telling no one. If it works great, I can release it. If it doesn’t, I can kill it and move on without having to eat crow, or get told I need to figure out a way to make it work after I’ve seen it’s not wise to continue.
I'm jealous that you have such flexibility at work that you can do POCs behind closed doors!
Nah, that's easy to solve.
"spend 3 hours on investigating X, 3 on Y, 3 on Z"; and then have a planning session about what to do next.
You have 4 outcomes - the first one solves it, the second, the third or none.
If it is solvable with mild effort, you have a 50% chance of solving it by attempt 2/6 hours.
"spend 3 hours on investigating X, 3 on Y, 3 on Z"; and then have a planning session about what to do next.
You have 4 outcomes - the first one solves it, the second, the third or none.
If it is solvable with mild effort, you have a 50% chance of solving it by attempt 2/6 hours.
It sounds like a pretty small problem if you can build a PoC in 3 hours. Sometimes it takes 3 days to get to a point where you have a rough idea of what code changes might be needed.
This gets a lot of teachers - the task is so ingrained in them that the only part they consciously understand is the trivial bit that everyone already gets. Breaking down a task and estimation are almost equivalent - once the breakdown is done, assigning estimates to the chinks of work takes a few minutes as anyone with a few years experience has some standard estimates for small tasks. In my experience I break down a task by biting off chunks that I can estimate, then ask for opinions on anything I can't.
Although I don't even believe the real path to mastery is the task breakdown. Anyone can come up with a bad breakdown. The real point of mastery, which he didn't talk about here, is identifying when the evidence suggests a task breakdown is incorrect enough to cause problems and communicating that / doing a re-estimate [0]. And being comfortable that it will happen so not getting stressed up front putting out a schedule that is likely to change. Managers generally want an accurate roadmap up front. This is them asking for the impossible. IMO Good management is about flexibility and understanding that the expected nature of the work changes as time passes and developers learn.
He has a little example at the end. Ponder that if someone had come to him with an accurate estimate ("you can do this in a few evenings as a plane trip") he'd have rejected that as too risky. That illustrates that estimating isn't even about accuracy, there are many unspoken factors here around risk management, expectation management, task familiarity, etc, etc.
[0] https://jacobian.org/2021/jun/8/incorrect-estimates/ - he has a post on the topic
Although I don't even believe the real path to mastery is the task breakdown. Anyone can come up with a bad breakdown. The real point of mastery, which he didn't talk about here, is identifying when the evidence suggests a task breakdown is incorrect enough to cause problems and communicating that / doing a re-estimate [0]. And being comfortable that it will happen so not getting stressed up front putting out a schedule that is likely to change. Managers generally want an accurate roadmap up front. This is them asking for the impossible. IMO Good management is about flexibility and understanding that the expected nature of the work changes as time passes and developers learn.
He has a little example at the end. Ponder that if someone had come to him with an accurate estimate ("you can do this in a few evenings as a plane trip") he'd have rejected that as too risky. That illustrates that estimating isn't even about accuracy, there are many unspoken factors here around risk management, expectation management, task familiarity, etc, etc.
[0] https://jacobian.org/2021/jun/8/incorrect-estimates/ - he has a post on the topic
Really great, honest and refreshing link - my thanks for posting it.
A lot of the "never taskers" in this comment section seemed to have never worked with the juniors I have.
These are good, new software folks who genuinely don't know how to stand up basic functionality because the space is new to them. In my experience they want tasks they can learn from and excel at, leading to growth.
I think the thing that you absolutely must keep in mind is process overhead is a continuum you tune to your team. No one in the NBA is going into a huddle and getting instruction on how to throw a ball mid game, but the kids in third grade are. Both styles of planning and coaching are appropriate to the team at hand.
These are good, new software folks who genuinely don't know how to stand up basic functionality because the space is new to them. In my experience they want tasks they can learn from and excel at, leading to growth.
I think the thing that you absolutely must keep in mind is process overhead is a continuum you tune to your team. No one in the NBA is going into a huddle and getting instruction on how to throw a ball mid game, but the kids in third grade are. Both styles of planning and coaching are appropriate to the team at hand.
I don’t know.
I think that’s the main truth about software engineering. Maybe an open source version of that Streak app exists and OP is re-inventing the wheel.
I think the brain doesn’t like task list. It may be pleasant to do, but most of entrepreneurship or coding is exploring.
And a task list blocks you from exploring.
I think that’s the main truth about software engineering. Maybe an open source version of that Streak app exists and OP is re-inventing the wheel.
I think the brain doesn’t like task list. It may be pleasant to do, but most of entrepreneurship or coding is exploring.
And a task list blocks you from exploring.
I'm curious: if you hired a contractor to, I dunno, paint your walls, would you accept "I don't know" as a time frame or price quote? If not, what's different about software development that makes "I don't know" a reasonable answer in our profession?
My answer to this is that software engineering is a lot more like creating a blueprint than creating an artifact based on that blueprint. Or put another way, it's usually more of a design process than a manufacturing process. (I've switched analogies on you, but "painting walls" in your analogy is akin to "manufacturing artifacts" in mine.)
The "manufacturing the artifact" step in software is done automatically by the computer, when given the "blueprint" (code).
I guess to try to go back to your painting walls analogy. In my view, creating software is more like if someone asked you to create a wall-painting machine that will work for any room that fits a certain specification. They could contract you to paint just one room in this way, but that would certainly be harder than just painting the room! But more likely, they have a million rooms they want you to paint. Either way, the hard part is creating the room-painting machine. And it would indeed be quite difficult to give an accurate estimate of how long it will take to do that. But once that exists, you can easily estimate how long it will take to paint any individual room.
And real engineers, indeed, have this same issue with the design phase of projects, for the same reasons. Just like us, they try to break apart and estimate how long it will take to do that part, but just like us, my impression is that it is known to be fraught to accurately estimate that part of a project.
But also just like us, this is a continuum based on how novel the thing they are engineering is to them. On the one end of the spectrum, there is the equivalent of off-the-shelf software, like using a blueprint for a standard single-family home that's been built a million times already. And on the other end of the spectrum are things like designing the motor for the first model of a new electric vehicle manufacturer, where it's all brand new. But then there's a whole spectrum between those two, where it is fairly easy to break down and estimate the design process for things you've done a bunch of times, and nearly impossible for things you've never done.
This is a very long-winded way of saying that the difference comes down to how novel the project you're working on is! This is why freelance / contractor shops do best when they find a particular niche of a kind of thing to build, and then find clients who want them to build essentially the same thing over and over again. It really is possible to get very good at estimating this kind of nearly-cookie-cutter work. (I did this a bit for awhile back in the day, with multi-page Rails CRUD apps, and it was indeed easy to break things up and estimate.) But this is also why it can be frustrating to work with freelance / contract shops, because it behooves them to figure out how to fit your project into their cookie cutter, and that can end up being a worse outcome than building something bespoke, iteratively, without a detailed plan and estimate.
The "manufacturing the artifact" step in software is done automatically by the computer, when given the "blueprint" (code).
I guess to try to go back to your painting walls analogy. In my view, creating software is more like if someone asked you to create a wall-painting machine that will work for any room that fits a certain specification. They could contract you to paint just one room in this way, but that would certainly be harder than just painting the room! But more likely, they have a million rooms they want you to paint. Either way, the hard part is creating the room-painting machine. And it would indeed be quite difficult to give an accurate estimate of how long it will take to do that. But once that exists, you can easily estimate how long it will take to paint any individual room.
And real engineers, indeed, have this same issue with the design phase of projects, for the same reasons. Just like us, they try to break apart and estimate how long it will take to do that part, but just like us, my impression is that it is known to be fraught to accurately estimate that part of a project.
But also just like us, this is a continuum based on how novel the thing they are engineering is to them. On the one end of the spectrum, there is the equivalent of off-the-shelf software, like using a blueprint for a standard single-family home that's been built a million times already. And on the other end of the spectrum are things like designing the motor for the first model of a new electric vehicle manufacturer, where it's all brand new. But then there's a whole spectrum between those two, where it is fairly easy to break down and estimate the design process for things you've done a bunch of times, and nearly impossible for things you've never done.
This is a very long-winded way of saying that the difference comes down to how novel the project you're working on is! This is why freelance / contractor shops do best when they find a particular niche of a kind of thing to build, and then find clients who want them to build essentially the same thing over and over again. It really is possible to get very good at estimating this kind of nearly-cookie-cutter work. (I did this a bit for awhile back in the day, with multi-page Rails CRUD apps, and it was indeed easy to break things up and estimate.) But this is also why it can be frustrating to work with freelance / contract shops, because it behooves them to figure out how to fit your project into their cookie cutter, and that can end up being a worse outcome than building something bespoke, iteratively, without a detailed plan and estimate.
Software development isn't (universally) a painting type task. In painting a room, the painter can estimate (by measuring) the amount of paint needed, they know from experience the time to acquire a particular color (if it needs to be mixed or can be bought readymade) in a particular volume, they know how long it takes to paint (they've done it many times before), how long to dry, and how many coats are needed for the type of surface.
Now, ask them to paint a mural instead. The estimates will change, they'll probably give you a broader range instead of being able to estimate almost down to the minute (and being off by no more than an hour or two) like someone just painting walls (with a known environment, surface type, and paint material).
Does the mural need to be designed? Are the desired qualities of the mural well-specified or will there be a series of back and forths? Maybe a set of prototypes (sketches) so that you can refine your requirements. At that point, the estimate for the total task (design and paint a mural, or design and develop a software application) becomes far less clear.
There are certainly programming tasks which are closer to the paint-a-wall task which are much easier to estimate reliably, but they're far from the only thing people in this field work on.
Now, ask them to paint a mural instead. The estimates will change, they'll probably give you a broader range instead of being able to estimate almost down to the minute (and being off by no more than an hour or two) like someone just painting walls (with a known environment, surface type, and paint material).
Does the mural need to be designed? Are the desired qualities of the mural well-specified or will there be a series of back and forths? Maybe a set of prototypes (sketches) so that you can refine your requirements. At that point, the estimate for the total task (design and paint a mural, or design and develop a software application) becomes far less clear.
There are certainly programming tasks which are closer to the paint-a-wall task which are much easier to estimate reliably, but they're far from the only thing people in this field work on.
This kind of thing destroys freelance devs who don't have context.
Customer: "Hi we need this small change." Dev: "No problem! Here's a quote for 4 hours"
Ignoring that 4 hours is probably less than the effort to consult, estimate, generate and process contracts, bill, do taxes... Then the customer comes back and asks for a revision which will at least require a Change-Order or a separate PO.
To some extent the overhead should be covered under labor burden but the reality is that in a sole proprietorship YOU are also the person doing the labor that is within the definition of "labor burden". It adds up extremely fast.
A million dollars would be a windfall if it landed in my checking account. On a development project with hardware and more than one dev it might as well be dust in the wind.
Customer: "Hi we need this small change." Dev: "No problem! Here's a quote for 4 hours"
Ignoring that 4 hours is probably less than the effort to consult, estimate, generate and process contracts, bill, do taxes... Then the customer comes back and asks for a revision which will at least require a Change-Order or a separate PO.
To some extent the overhead should be covered under labor burden but the reality is that in a sole proprietorship YOU are also the person doing the labor that is within the definition of "labor burden". It adds up extremely fast.
A million dollars would be a windfall if it landed in my checking account. On a development project with hardware and more than one dev it might as well be dust in the wind.
Contractor horror stories are actually very common. They may not actually say to you "I don't know", but big delays in contractor jobs happen all the time. I would argue that software engineers are at least trying to be more honest about the unknowns of the work unlike some sleazy contractors.
> what's different about software development that makes "I don't know" a reasonable answer in our profession?
The seemingly infinite amount of variations in software tasks and the ambiguity of the requirements? If my job involved only putting up or modifying API endpoints that involved querying a single SQL database, I'd get quite good at that and be able to tell you with very good accuracy how long an endpoint would take to put up.
Just as if I painted many walls before, I could reasonably tell you how long painting a wall would take based on the surface area, height, and amount of non-wall obstructions. Also, contractors tend to be their own salesmen, so just because they speak confidently, doesn't mean they are. It's not like contractors are universally famous for being on-time and underbudget.
The seemingly infinite amount of variations in software tasks and the ambiguity of the requirements? If my job involved only putting up or modifying API endpoints that involved querying a single SQL database, I'd get quite good at that and be able to tell you with very good accuracy how long an endpoint would take to put up.
Just as if I painted many walls before, I could reasonably tell you how long painting a wall would take based on the surface area, height, and amount of non-wall obstructions. Also, contractors tend to be their own salesmen, so just because they speak confidently, doesn't mean they are. It's not like contractors are universally famous for being on-time and underbudget.
If your job involved _only_ putting up or modifying API endpoints that involved querying a single SQL database, you could automate at least a good chunk of that. Maybe all of it.
How long would it take to build a system to automate API endpoints that query a single SQL database? It depends.
How long would it take to build a system to automate API endpoints that query a single SQL database? It depends.
Agree, the entire point of a task list is to block yourself from doing other things. That makes sense until it doesn't. "The map is not the territory".
The biggest problem I have with splitting tasks is reluctance to do duplicate or unnecessary work. Breaking down tasks into smaller tasks means you must do duplicate work. The trick is to minimize it.
If you try to make no unnecessary work, you'll end up having to do everything at once.
Example: you want to refactor a program with two modules A and B where B depends on A. The least wasteful way of doing it is to refactor both modules together. But it's also the riskiest and hardest to estimate.
The broken down way is to refactor A and adapt B so that it works with the refactored A. After that you would refactor A, which would then risk making the adaptation effort very short lived.
If you want to do zero throwaway effort, you often can't break down the task. I have 20 years of experience and I still often find myself reluctant to do throwaway/temporary job in order to divide work. Instead I find myself doing multi-week efforts with zero yaks left unshaved.
If you try to make no unnecessary work, you'll end up having to do everything at once.
Example: you want to refactor a program with two modules A and B where B depends on A. The least wasteful way of doing it is to refactor both modules together. But it's also the riskiest and hardest to estimate.
The broken down way is to refactor A and adapt B so that it works with the refactored A. After that you would refactor A, which would then risk making the adaptation effort very short lived.
If you want to do zero throwaway effort, you often can't break down the task. I have 20 years of experience and I still often find myself reluctant to do throwaway/temporary job in order to divide work. Instead I find myself doing multi-week efforts with zero yaks left unshaved.
Maybe I’m just lazy/undisciplined/a cowboy but having to break things down into “pointable” tasks feels like busy work just so management can “see” progress.
I mean, thinking through a problem you’re trying to solve before starting makes sense, and having rough milestones is important - but a majority of the time there’s so many unknown unknowns that fully breaking it down is totally useless, if not impossible.
If I spent the same amount of time it took me to break a project down just working on figuring out a solution or building it, it’d get done way sooner. At least, that’s how it feels.
I mean, thinking through a problem you’re trying to solve before starting makes sense, and having rough milestones is important - but a majority of the time there’s so many unknown unknowns that fully breaking it down is totally useless, if not impossible.
If I spent the same amount of time it took me to break a project down just working on figuring out a solution or building it, it’d get done way sooner. At least, that’s how it feels.
I mostly disagree with you if you work in a "standard" company - it's usually "make a form" or "move data / do some other CRUD". These tasks don't generally have that many unknowns.
It's absolutely valuable for managers to be able to estimate velocity as well, though the problem is that they really do stop understanding "we are not sure" once you spoil them.
I am a consultant too, so it's critical for me to say "we are delivering this, in this much time", even though we are "agile".
It's absolutely valuable for managers to be able to estimate velocity as well, though the problem is that they really do stop understanding "we are not sure" once you spoil them.
I am a consultant too, so it's critical for me to say "we are delivering this, in this much time", even though we are "agile".
> feels like busy work just so management can “see” progress
Lot's of people feel like this. I usually come at it a different way - we're trying to estimate effort to see if we want to attempt building it in the first place. Assisting with the cost-benefit question if you will. That's the first reason.
It can also be extremely beneficial to the team, particularly when working with less experienced folks. You can farm out a lot of the work and parallelise the task.
I suspect Jacob didn't do much of this breakdown when implementing his streak app, which is why he recurses in an illustrative way.
Lot's of people feel like this. I usually come at it a different way - we're trying to estimate effort to see if we want to attempt building it in the first place. Assisting with the cost-benefit question if you will. That's the first reason.
It can also be extremely beneficial to the team, particularly when working with less experienced folks. You can farm out a lot of the work and parallelise the task.
I suspect Jacob didn't do much of this breakdown when implementing his streak app, which is why he recurses in an illustrative way.
Ignoring management part, one place where I have found breaking down to be helpful is when I have to work on some thing which I am not terribly motivated todo (say it is boring, lethargy, or it looks too daunting). In such cases I have found it useful to break down to smaller tasks and keep completing each of them which in turn generates a momentum to continue, as I have been able to make some progress from a situation where I was essentially in a deadlock (or was lethargic to work on).
The best way to estimate is to start work and benchmark your progress against your estimates
Another post has this part: https://jacobian.org/2021/may/25/my-estimation-technique/#5-...
Does anyone have a better article on this? This is ok, but it seems like there are better - I would love to share with my team.
These are things I always share with them:
On code reviews: https://mtlynch.io/code-review-love/
On simplicity: https://grugbrain.dev/
On SOLID (even though we use Python now): https://www.baeldung.com/solid-principles
I would love to have a "standard" article for planning.
These are things I always share with them:
On code reviews: https://mtlynch.io/code-review-love/
On simplicity: https://grugbrain.dev/
On SOLID (even though we use Python now): https://www.baeldung.com/solid-principles
I would love to have a "standard" article for planning.
[deleted]
Sunk cost fallacy can be a problem if you spent sufficient time breaking down tasks for a big project. You may not want to pivot and re breakdown
Isn't it more likely that you can catch yourself during the planning phase if the amount of work being estimated exceeds your budget? That seems much cheaper and easier to pivot from than if you were to make that realization halfway through your implementation.
I think it’s par for the course because you can only plan so far before you run into miscalculations once you start doing it. You break down tasks for stuff very close range
> Sunk cost fallacy can be a problem if you spent sufficient time breaking down tasks for a big project. You may not want to pivot and re breakdown
Similarly, if you don't plan enough, and get far enough into a small project, you don't change your approach because of the sunk cost fallacy.
Similarly, if you don't plan enough, and get far enough into a small project, you don't change your approach because of the sunk cost fallacy.
Right, you are susceptible to it either way. In that case breaking down a task can potentially get you way further
This reminds me a bit of the old "Write a story about the task. The nouns are objects, and the verbs are methods." method of OOP.
I never warmed to that.
However, this is not that, and I think it's an important skill. I started my professional life as an RF technician, and we learned how to do this, almost immediately. We used things like signal generators and oscopes.
I never warmed to that.
However, this is not that, and I think it's an important skill. I started my professional life as an RF technician, and we learned how to do this, almost immediately. We used things like signal generators and oscopes.
What's missing here is who this breakdown should be shared with. There are three levels of breakdown and the details of lower levels should never be revealed to higher levels:
1. The "business" level. There are no "tasks". There are only needs and desires. Things like "a user can log in and press a button". These should not be broken down further than the smallest unit of deliverable value. In this example, there's no point estimating "user can log in" because it delivers no value on its own. A good rule of thumb is these should be described using descriptive language, not imperative, so not e.g. "implement log in procedure".
2. The "team" level. It's OK here to break down those things into "user can log in" and "logged in user can press button". That's because you know they can be implemented independently. But there's still no point delivering them independently. Don't tell the upper level about this breakdown. They are always really eager to know, but they don't need to know. Use this to calculate your estimate for level 1, but said estimate should be a single aggregate. Implementing these in parallel with multiple devs adds no extra overhead because they are completely independent.
3. The "developer" level. This is where you finally have "tasks" and imperative language. Things like "add button to form", "implement 2FA" etc. These tasks are naturally heavily dependent on each other and it's highly likely they'll be done in a completely different order to whatever you write them down in.
If you decide to distribute these tasks among developers then you increase the coordination overhead between devs. Think of it like a multi-threaded application. It's always more efficient for a single developer to do everything if possible (less overhead), but it might nevertheless be necessary to split it up due to time constraints, differing skillsets etc. It's just like we'd like to have one single 80GHz general purpose CPU, but in reality we have to make do with 16 5GHz cores split between performance and efficiency cores etc.
Given that the tasks are highly likely to change and evolve, there is not much point putting in loads of effort to break things down. Do it only when it's necessary, or when it helps you. It's necessary when you need to split up the work between developers. It's helpful when you think of more tasks during your work and you don't want to break your stack. Every developer should have a way to quickly take notes in a way that doesn't break flow, things like "ensure API accepts float input". This is stuff you'd never think of before you get your teeth into it.
1. The "business" level. There are no "tasks". There are only needs and desires. Things like "a user can log in and press a button". These should not be broken down further than the smallest unit of deliverable value. In this example, there's no point estimating "user can log in" because it delivers no value on its own. A good rule of thumb is these should be described using descriptive language, not imperative, so not e.g. "implement log in procedure".
2. The "team" level. It's OK here to break down those things into "user can log in" and "logged in user can press button". That's because you know they can be implemented independently. But there's still no point delivering them independently. Don't tell the upper level about this breakdown. They are always really eager to know, but they don't need to know. Use this to calculate your estimate for level 1, but said estimate should be a single aggregate. Implementing these in parallel with multiple devs adds no extra overhead because they are completely independent.
3. The "developer" level. This is where you finally have "tasks" and imperative language. Things like "add button to form", "implement 2FA" etc. These tasks are naturally heavily dependent on each other and it's highly likely they'll be done in a completely different order to whatever you write them down in.
If you decide to distribute these tasks among developers then you increase the coordination overhead between devs. Think of it like a multi-threaded application. It's always more efficient for a single developer to do everything if possible (less overhead), but it might nevertheless be necessary to split it up due to time constraints, differing skillsets etc. It's just like we'd like to have one single 80GHz general purpose CPU, but in reality we have to make do with 16 5GHz cores split between performance and efficiency cores etc.
Given that the tasks are highly likely to change and evolve, there is not much point putting in loads of effort to break things down. Do it only when it's necessary, or when it helps you. It's necessary when you need to split up the work between developers. It's helpful when you think of more tasks during your work and you don't want to break your stack. Every developer should have a way to quickly take notes in a way that doesn't break flow, things like "ensure API accepts float input". This is stuff you'd never think of before you get your teeth into it.
Can you elaborate on why we wouldn't share the carefully estimated project plan breakdown with Level 1? I'm not disagreeing, just curious - one would think that with doing all the work to achieve a solid project plan breakdown and Gantt charts and such, it would behoove you to express this to management who is salivating over tangible examples of progress... I understand we don't want to show technical to non-technical, but part of the good project plan is the fact that we can show a roll-up of the tasks, so the abstraction is visible. It's hierarchical.
Reporting on progress up to level 1 is a waste of everyone's time. It's either done or it's not. It ships when it ships. If the team has committed to delivering by a date then just let them get on with doing that. Blockers and setbacks should be actively communicated up the stack if they will affect the deliverable, but otherwise it should be assumed things are on track.
Management are always salivating for details, but we know what always happens: we drop a technical term one time and management goes around repeating that like they know what they're talking about. They use these like weapons to keep other people off their back: basically say something that nobody understands and they'll leave you to it. Some managers will push for more fodder, but it's not your job to provide this. It's your job to deliver.
Management are always salivating for details, but we know what always happens: we drop a technical term one time and management goes around repeating that like they know what they're talking about. They use these like weapons to keep other people off their back: basically say something that nobody understands and they'll leave you to it. Some managers will push for more fodder, but it's not your job to provide this. It's your job to deliver.
I created tasktree.co for myself to help with this! It’s so easy to get overwhelmed when faced with a big task, and actually forcing yourself to just break it down helps a lot.
Meta: any other blogs like this one? I really enjoy the writing, the topics..
Good read! I wish we had a shorter name for "breaking down tasks". Perhaps taskspec, short for "task specifying". Or taskdev?
"refinement" in JIRA world I think
[deleted]
One, it turns out I never ever end up doing the actual steps as I planned to. After a few I have new insights, realize I forgot something, see an easier way, I don't know -- the plan is never followed.
The other is that I really hate working like this because it seems all the creative effort of thinking about how to make something is put in the first part. And then, the rest is still most of the work, but it's the really boring part. It's much more fun (and therefore faster, and resulting in better work) to spread the creativity and the boredom out more equally.
The two are probably related, and it's not inconceivable that I have ADHD.