The non-linearity of productivity(blog.kenperlin.com)
blog.kenperlin.com
The non-linearity of productivity
http://blog.kenperlin.com/?p=18993
80 comments
> Steve McConnell in Code Complete (a book which criticizes the widespread use of the waterfall model) refers to design as a “wicked problem” – a problem whose requirements and limitations cannot be entirely known before completion. The implication is that it is impossible to get one phase of software development “perfected” before time is spent in “reconnaissance” working out exactly where and what the big problems are.
https://en.wikipedia.org/wiki/Wicked_problem
https://en.wikipedia.org/wiki/Wicked_problem
I used to be an architect before I became a programmer, so since I started programming I noticed I hag a big advantage over my peers when it comes to the high level designing and structuring a program.
Thing is, the more experienced I become at programming, the less of that I end up doing. I more and more tend to break everything down into tiny parts that can all work independently and be composed into bigger bits. Most of my career I've used OO languages, especially C#, but nowadays I usually avoid using a lot of the features of OO, like inheritance, and just try and make everything from tiny little methods (functions) big things are just made from several of those methods in sequence and really big things are just made from several of the big ones.
From a development perspective it means I can just go ahead and start writing useful little snippets of code and start composing them together into bigger and bigger features as I go. So, I'm switching from a top-down style to more of a bottom-up one.
Thing is, the more experienced I become at programming, the less of that I end up doing. I more and more tend to break everything down into tiny parts that can all work independently and be composed into bigger bits. Most of my career I've used OO languages, especially C#, but nowadays I usually avoid using a lot of the features of OO, like inheritance, and just try and make everything from tiny little methods (functions) big things are just made from several of those methods in sequence and really big things are just made from several of the big ones.
From a development perspective it means I can just go ahead and start writing useful little snippets of code and start composing them together into bigger and bigger features as I go. So, I'm switching from a top-down style to more of a bottom-up one.
It's always interesting to me to see how various people discover the basic advantage of function-oriented programming on their own.
There's also eg the copy-on-write filesystems, and git---whose main on-disk data-structures are conceptually immutable despite being implemented in C.
You might enjoy reading "Out of the tarpit" (https://github.com/papers-we-love/papers-we-love/blob/master...), a paper that deals with approaches to master software complexity.
There's also eg the copy-on-write filesystems, and git---whose main on-disk data-structures are conceptually immutable despite being implemented in C.
You might enjoy reading "Out of the tarpit" (https://github.com/papers-we-love/papers-we-love/blob/master...), a paper that deals with approaches to master software complexity.
TL;DR Composition vs Inheritance
Agreed. And I'm increasingly tending to think that the stuck-thinking-about-the-tricky-stuff phase is the part of the game I find most satisfying.
A lot of textbook management techniques seem hellbent on trying to force steady productivity instead of punctuated equilibrium. Not the recipe for a satisfying environment.
A lot of textbook management techniques seem hellbent on trying to force steady productivity instead of punctuated equilibrium. Not the recipe for a satisfying environment.
I once had a manager ask me to email him a progress report every day; and every hour or two, he'd make a point of interrupting me to ask me "so how's <current_task> going?"
Sounds like something straight of office space, but that was my life for a while. Least productive job I've ever had because of that (well, and the myriad other little toxic/passive aggressive behaviors he had).
He had a Stanford MBA too, which convinced me of how useless MBAs must be. Because if you make it through one of the top programs in "business administration" in the world and still manage like that, you might as well have an MD from Harvard and still believe bloodletting is a viable treatment for illnesses.
Sounds like something straight of office space, but that was my life for a while. Least productive job I've ever had because of that (well, and the myriad other little toxic/passive aggressive behaviors he had).
He had a Stanford MBA too, which convinced me of how useless MBAs must be. Because if you make it through one of the top programs in "business administration" in the world and still manage like that, you might as well have an MD from Harvard and still believe bloodletting is a viable treatment for illnesses.
I once had a manager ask me to email him a progress report every day;
This seems to have become very nearly normalised in the guise of daily stand ups. Some would argue it's less intrusive if it's for "the team" rather than for an individual manager, but I'm not convinced myself.
and every hour or two, he'd make a point of interrupting me to ask me "so how's <current_task> going?"
Similarly, in environments where tasks are getting broken down into tickets that are only expected to take a few hours, there's a similar level of oversight (although hopefully a bit less disruptive than hourly shoulder-taps, I agree).
This seems to have become very nearly normalised in the guise of daily stand ups. Some would argue it's less intrusive if it's for "the team" rather than for an individual manager, but I'm not convinced myself.
and every hour or two, he'd make a point of interrupting me to ask me "so how's <current_task> going?"
Similarly, in environments where tasks are getting broken down into tickets that are only expected to take a few hours, there's a similar level of oversight (although hopefully a bit less disruptive than hourly shoulder-taps, I agree).
I worked for a startup that grew up around me. We started without any project managers and such and they got introduced as the team grew and grew.
I would just make tasks like "figure out how to do X", "look into this framework Y", "write a really good SO post for the bug we came across in library Z" . Worked fine, tho the PM for my project was sound.
I'm at the first job I've ever had that didn't do daily standups and it's liberating. I've never been more productive at work.
Maybe once a week my manager comes by and asks me how something is going. I give him a high level sketch and a time estimate. He goes along on his way after that.
Maybe once a week my manager comes by and asks me how something is going. I give him a high level sketch and a time estimate. He goes along on his way after that.
He's treating you like a cog in a manufacturing plant, which he can "optimize" at will. MBA programs often teach people that. He doesn't understanding creative work or thinking heavy work "appear" to be non-working but it's essential.
Why does asking how someone is doing mean you're a cog, or that you're in trouble if you're still doing research?
Part of what a supervisor does is track progress across a whole team so they can keep other parts of the group appraised.
Another important job is helping people make tradeoffs between perfect and done as the situation on the ground changes.
Both of those require asking "how's it going?"
Part of what a supervisor does is track progress across a whole team so they can keep other parts of the group appraised.
Another important job is helping people make tradeoffs between perfect and done as the situation on the ground changes.
Both of those require asking "how's it going?"
> I once had a manager ask me to email him a progress report every day; and every hour or two, he'd make a point of interrupting me to ask me "so how's <current_task> going?"
Asking a developer's status progress everyday or every hour? That shows the manager has absolutely no clue how development works. Period. Developers don't need a manager to "help" them make tradeoff on a daily basis. Developers need managers to back the fuck off to let them alone to concentrate on the mentally taxing work at hand.
Asking a developer's status progress everyday or every hour? That shows the manager has absolutely no clue how development works. Period. Developers don't need a manager to "help" them make tradeoff on a daily basis. Developers need managers to back the fuck off to let them alone to concentrate on the mentally taxing work at hand.
Here are some possible explanation for the behaviour. There might be other urgent fires burning for the past week and he is getting nervous and trying to figure out whether he needs to reschedule. He also has to manage up and set expectations when the whole project is going to be done. He might be concerned you are gold plating an otherwise simple task. He might think you are stuck and trying to figure things out on your own. You could help him by letting him have a sense of where you are, and whether there is something more pressing that he needs to talk about. Perhaps he has access to people/equipment/money that can accelerate your work.
The manager in question, were he a good one, should also be able to clearly and timely communicate if any or all of these things apply.
I guess the MBA is there to facilitate your professional development, not to make you more productive.
And what you have to do in order to look good to your manager or manager's manager's manager is different than what you have to do to be productive. Your boss was perhaps just doing the former?
And what you have to do in order to look good to your manager or manager's manager's manager is different than what you have to do to be productive. Your boss was perhaps just doing the former?
"A lot of textbook management techniques..."
Can you unpack this? Would like to hear more.
Can you unpack this? Would like to hear more.
I have "management" in my degree title, from a world famous uni. Basically a good taste of an MBA, since an MBA there is a year.
Very little of a management course is about how you treat people. If anything, you are left with Taylorism and the lingering question of how it relates to knowledge workers. Because Taylor was basically measuring factory workers' performance, whose contribution is to do some repeated action over and over. If that sounds like treating someone like a robot, it's because it is.
This is not to say nobody has thought about this. HBR will run an article now and again about HR issues. But there's nothing especially compelling where you'd be able to find a root cause, find a specific solution, and implement it. If it was, management wouldn't be magic, it would be engineering. And the practitioners would not be paid enormous sums for doing the impossible, they'd be paid moderately well for being ordinarily competent.
Most of the things on the course are really not that different from reading the FT regularly. So-and-so company found their product wasn't appealing to young people, so they tried X and it was a massive success. Doesn't tell you what to do if you were to run this firm, but after you've read a few you'll think you know. Case studies are a case study in post-hoc fallacy.
Then there's a bunch of supposedly advanced financial stuff, which if you're an engineer you'll laugh at. Ratios, really? Interest rates? If you do a proper finance course there will be real things to learn (stoch calc doesn't show up much in engineering), but not in the business school.
And there's a few useful things, too, like the history of capitalism, but nothing that develops a proper intuition or method for a field of problems like engineering does.
Very little of a management course is about how you treat people. If anything, you are left with Taylorism and the lingering question of how it relates to knowledge workers. Because Taylor was basically measuring factory workers' performance, whose contribution is to do some repeated action over and over. If that sounds like treating someone like a robot, it's because it is.
This is not to say nobody has thought about this. HBR will run an article now and again about HR issues. But there's nothing especially compelling where you'd be able to find a root cause, find a specific solution, and implement it. If it was, management wouldn't be magic, it would be engineering. And the practitioners would not be paid enormous sums for doing the impossible, they'd be paid moderately well for being ordinarily competent.
Most of the things on the course are really not that different from reading the FT regularly. So-and-so company found their product wasn't appealing to young people, so they tried X and it was a massive success. Doesn't tell you what to do if you were to run this firm, but after you've read a few you'll think you know. Case studies are a case study in post-hoc fallacy.
Then there's a bunch of supposedly advanced financial stuff, which if you're an engineer you'll laugh at. Ratios, really? Interest rates? If you do a proper finance course there will be real things to learn (stoch calc doesn't show up much in engineering), but not in the business school.
And there's a few useful things, too, like the history of capitalism, but nothing that develops a proper intuition or method for a field of problems like engineering does.
This just, kind of blows my mind. What is management if not people? I mean Drucker's very definition of management is the idea of organizing a group of people to drive towards a common aim where you learn how to make each individuals' strengths effective and their weaknesses irrelevant.
Deming was adamant that psychology had to be a pillar of management. Why? Because people!
I'm curious then, who were the authorities/what were the theories (if any I guess) that were used for your management degree?
- Also, why am I talking about Drucker and Deming? Because if I drew a family tree, I could trace at least some aspect of nearly any management idea today back to them (lots of amalgams out there these days). There is one big caveat though, lots of management practices today have neo-Taylorism embedded into them that most people are unaware of.
Deming was adamant that psychology had to be a pillar of management. Why? Because people!
I'm curious then, who were the authorities/what were the theories (if any I guess) that were used for your management degree?
- Also, why am I talking about Drucker and Deming? Because if I drew a family tree, I could trace at least some aspect of nearly any management idea today back to them (lots of amalgams out there these days). There is one big caveat though, lots of management practices today have neo-Taylorism embedded into them that most people are unaware of.
> I'm curious then, who were the authorities/what were the theories (if any I guess) that were used for your management degree?
It felt like a very sporadic hodge podge of names. Porter's five forces, tech cycle, a load of case studies.
That's looking back a decade later, so maybe I'm missing something.
But it all seemed very vague. I remember thinking how on earth did Porter get his name on a "theory" when really that's just common sense? A company is going to tend to have customers, suppliers, entrants, and substitutes, and internal rivalry. I mean why would you think otherwise? Also, creating those categories is no guarantee that you're not going to miss out something.
If anything I've learned more about management from reading agile type stuff like Scrum, because those kinds of things are usually grounded in experience from a specific industry, with specific pain points and specific solutions.
It felt like a very sporadic hodge podge of names. Porter's five forces, tech cycle, a load of case studies.
That's looking back a decade later, so maybe I'm missing something.
But it all seemed very vague. I remember thinking how on earth did Porter get his name on a "theory" when really that's just common sense? A company is going to tend to have customers, suppliers, entrants, and substitutes, and internal rivalry. I mean why would you think otherwise? Also, creating those categories is no guarantee that you're not going to miss out something.
If anything I've learned more about management from reading agile type stuff like Scrum, because those kinds of things are usually grounded in experience from a specific industry, with specific pain points and specific solutions.
> I remember thinking how on earth did Porter get his name on a "theory" when really that's just common sense?
This is how I've always felt about "De Morgan's Laws". There are two, and they state:
1. If it is not the case that two things are both true, then at least one of those things is false.
2. If it is not the case that either of two things is true, then both of those things are false.
(Actually, they are logical equivalences, so they also apply in the other order -- "1b. If at least one of two things is false, then it is not the case that both things are true.")
I'm pretty sure both of those "laws" were known well before the nineteenth century.
This is how I've always felt about "De Morgan's Laws". There are two, and they state:
1. If it is not the case that two things are both true, then at least one of those things is false.
2. If it is not the case that either of two things is true, then both of those things are false.
(Actually, they are logical equivalences, so they also apply in the other order -- "1b. If at least one of two things is false, then it is not the case that both things are true.")
I'm pretty sure both of those "laws" were known well before the nineteenth century.
Thanks.
Ultimately, I think I was referring to the Taylorist aspect.
There's also a philosophy that I've seen labelled as "teamism" and which seems to form the basis of methodologies like Scrum. I'm not sure teamism is quite the same thing as Taylorism -- but it shares the same distrust for individuals working on problems without near-constant oversight.
Ultimately, I think I was referring to the Taylorist aspect.
There's also a philosophy that I've seen labelled as "teamism" and which seems to form the basis of methodologies like Scrum. I'm not sure teamism is quite the same thing as Taylorism -- but it shares the same distrust for individuals working on problems without near-constant oversight.
Scrum has elements of Taylorism: team members are treated as interchangeable cogs to be assigned units of work and it has a crappy "productivity" measure built in (velocity).
I don't measure velocity in the scrum team I coach. And people's skills are not interchangeable but we do want most of the team member to be able to perform the easy stuff. A front end developer is not interchangeable with a backend developer. Having a task card about thinking through a design is always welcome. We use standup to have face to face conversations.
> And the practitioners would not be paid enormous sums for doing the impossible, they'd be paid moderately well for being ordinarily competent.
That's an awesome description of where we should move management to, hopefully. (Same for programming, marketing etc.)
That's an awesome description of where we should move management to, hopefully. (Same for programming, marketing etc.)
[deleted]
"Textbook" was possibly more a nervous tic than a statement of fact -- I've not read a lot of management textbooks...
However, whenever I see generalist managers at work, there always seems to be massive emphasis on schedule risk and at least in organisation I've been in, the proposed mitigation is always regular status meetings (and perhaps other forms of check-in as well).
This seems to have been almost entirely normalised in the software world, particularly anywhere that's gone down the agile pathway: fine-grained tickets in an issue tracker (or possibly cards on the wall -- but haven't ever seen that one in action) and a daily standup with an explicit goal of making sure nobody is "spinning their wheels".
However, whenever I see generalist managers at work, there always seems to be massive emphasis on schedule risk and at least in organisation I've been in, the proposed mitigation is always regular status meetings (and perhaps other forms of check-in as well).
This seems to have been almost entirely normalised in the software world, particularly anywhere that's gone down the agile pathway: fine-grained tickets in an issue tracker (or possibly cards on the wall -- but haven't ever seen that one in action) and a daily standup with an explicit goal of making sure nobody is "spinning their wheels".
Thanks for the explanation. I've got a working theory: most people, even managers, even lauded ones, don't know much about "textbook" management. They mostly manage how they were managed, some picking up a few things here or there.
It's recursive.
I've worked in businesses, started and scaled my own, and have also been a teacher (k-12). We all go to school for at least 13 years and internalize what "teaching" is. Most teachers' classrooms end up looking like ones they experienced even if the "textbook" lays out massively different ideas. It's the same problem as management.*
We all also work, and internalize some conception of "management." Thus, most manage they way they were managed.
Throughout my career I've worked with over a hundred different organizations. I can read about many more in the news and case studies. I see very little in practice that aligns with hardly any of the big ideas in management that I've learned from the likes of Drucker and Deming.
*It has not gone unnoticed that education is our first experience with management. Often it has Taylorist, Theory X, command and control roots. In contrast, the great teachers I see have a lot in common with great managers and management theory (see Drucker, Deming).
It's recursive.
I've worked in businesses, started and scaled my own, and have also been a teacher (k-12). We all go to school for at least 13 years and internalize what "teaching" is. Most teachers' classrooms end up looking like ones they experienced even if the "textbook" lays out massively different ideas. It's the same problem as management.*
We all also work, and internalize some conception of "management." Thus, most manage they way they were managed.
Throughout my career I've worked with over a hundred different organizations. I can read about many more in the news and case studies. I see very little in practice that aligns with hardly any of the big ideas in management that I've learned from the likes of Drucker and Deming.
*It has not gone unnoticed that education is our first experience with management. Often it has Taylorist, Theory X, command and control roots. In contrast, the great teachers I see have a lot in common with great managers and management theory (see Drucker, Deming).
> However, whenever I see generalist managers at work, there always seems to be massive emphasis on schedule risk and at least in organisation I've been in, the proposed mitigation is always regular status meetings (and perhaps other forms of check-in as well).
I think the reason for that behavior is because they don't want to be caught unawares when some deadline is missed. In other words, one of the metrics they are measured on is meeting deadlines, which of course pushes schedule risk to the top of the pile. There's merit to that, but that has to be balanced with the quality of the product.
The blind spot for such managers is that constant check-ins and updates on progress too frequently interrupt the flow of people who need time to think things through. Adding pressure at the wrong point can possibly ruin the quality of the final product.
I think the reason for that behavior is because they don't want to be caught unawares when some deadline is missed. In other words, one of the metrics they are measured on is meeting deadlines, which of course pushes schedule risk to the top of the pile. There's merit to that, but that has to be balanced with the quality of the product.
The blind spot for such managers is that constant check-ins and updates on progress too frequently interrupt the flow of people who need time to think things through. Adding pressure at the wrong point can possibly ruin the quality of the final product.
I have a completely different way of mitigating schedule risk: find the most difficult, unpredictable subproblem and do that first. I guess it's not quite fair to say that I've never seen a manager pick up on this idea, but it still doesn't seem to have become their first-choice strategy.
An experienced architect will always do this. Often the most risky subproblem is the one that is least understood (unknown unknowns), which needs to be investigated, even if it turns out not to be a great risk.
In agile-speak this is called 'spike solution'.
Also see lean architecture methods, such as:
- 'Just Enough Software Architecture', by George Fairbanks.
- 'Risk and Cost Driven Architecture', by Eltjo Poort.
- 'Just Enough Software Architecture', by George Fairbanks.
- 'Risk and Cost Driven Architecture', by Eltjo Poort.
One problem is that 'agile' has been taken over by management types.
> Being stuck and being moving are dependent on each other.
Yeah, but in times of being stuck, it's much easier to get distracted and procrastinate.
Yeah, but in times of being stuck, it's much easier to get distracted and procrastinate.
> Yeah, but in times of being stuck, it's much easier to get distracted and procrastinate.
A bit, but I tend to do directed procrastination. Go the gym, do some research on a different problem, write some emails, etc... As soon as I stop actively thinking about the problem a solution will come.
Another solution (and this works well with coding and writing) is just to start coding/writing. I'm writing an article now for a newsletter, and the top of my document is a bunch of stand alone sentences that eventually lead to a full article.
In programming, I'll often do the same thing and just start solving the problem even if I know it's not the right solution yet. Usually this leads me to a better understanding, and eventually the solution.
A bit, but I tend to do directed procrastination. Go the gym, do some research on a different problem, write some emails, etc... As soon as I stop actively thinking about the problem a solution will come.
Another solution (and this works well with coding and writing) is just to start coding/writing. I'm writing an article now for a newsletter, and the top of my document is a bunch of stand alone sentences that eventually lead to a full article.
In programming, I'll often do the same thing and just start solving the problem even if I know it's not the right solution yet. Usually this leads me to a better understanding, and eventually the solution.
Directed procrastination works for me too. I usually hate admin type of work; sorting out my room, doing the laundry, responding in social media (yes, that counts too..)
But when I hit the inertia part of the productivity cycle, this saves me. It's always better to do something than just being stuck. And while I mechanically do those tasks, my head also gets in order, I start dreaming and slowly, I'm on the roll again.
When the right time comes, I slide back into the 'real' work. I can't quite regret for 'wasting' that time, because usually the reward is a new perspective, and the ideas that come with it.
Shrugs, we're not machines.
But when I hit the inertia part of the productivity cycle, this saves me. It's always better to do something than just being stuck. And while I mechanically do those tasks, my head also gets in order, I start dreaming and slowly, I'm on the roll again.
When the right time comes, I slide back into the 'real' work. I can't quite regret for 'wasting' that time, because usually the reward is a new perspective, and the ideas that come with it.
Shrugs, we're not machines.
> As soon as I stop actively thinking about the problem a solution will come.
The book 'Hare Brain, Tortoise Mind' has a good explanation of how that works, what it's good at and what it's not.
I get a lot of side chores done with directed procrastination. But, what works for me to get the main task done is to just open the IDE even without a goal. Instead of "Ugg... I need to make X work today...". Just open the editor and do anything. Refactor a function. Comment something. Whatever. Once you get started, it much easier to continue.
The book 'Hare Brain, Tortoise Mind' has a good explanation of how that works, what it's good at and what it's not.
I get a lot of side chores done with directed procrastination. But, what works for me to get the main task done is to just open the IDE even without a goal. Instead of "Ugg... I need to make X work today...". Just open the editor and do anything. Refactor a function. Comment something. Whatever. Once you get started, it much easier to continue.
My problem is when I open the editor and refactor a function, I start finding other functions that need to be refactored, then finding even more refactoring opportunities from those, recursively, forever. At some point I need to stop refactoring and start producing something new...
Some amount of distraction is probably helpful. Once you have gathered all requirements and have thought about the problem for a while it is good to let it marinate for some time. You will often have moments of insight seemingly out of nowhere while you are e.g. going out for a walk.
Agreed. I often figure out a solution to a problem I'm having as soon as I stand up from my desk!
The thing I find works better than anything else is to write some code that works, even though I know it's far from the "right" design.
Once you have concrete working code, it's much easier to organize it in ways that make sense than trying to visualize everything beforehand.
As a young programmer I was taught to carefully design code before starting to write it. It kinda blows my mind that now I design it after writing!
Once you have concrete working code, it's much easier to organize it in ways that make sense than trying to visualize everything beforehand.
As a young programmer I was taught to carefully design code before starting to write it. It kinda blows my mind that now I design it after writing!
That works for me too. I'm originally a lawyer and a decently competent writer and that's also how I approach papers or briefs. Writing is how I think and organize my thoughts so there's no more efficient way for me to design an overall architecture, whether in code or English, than just by starting to write. Once I start to get a grip on the parameters and pain points, I'll circle back and outline the higher level. My brain just works better from a bottom up starting point. Writing and coding are just formalized ways of thinking so sometimes you have to make concessions to your own brain's idiosyncrasies.
Yeah, I write a lot that way too, probably influenced by my programming habits.
I read somewhere that Neil Gaiman considers himself only a decent writer who is an extremely good editor. Which is to say, he starts writing a simple draft, and then changes what's wrong with it, over and over.
I read somewhere that Neil Gaiman considers himself only a decent writer who is an extremely good editor. Which is to say, he starts writing a simple draft, and then changes what's wrong with it, over and over.
bottom-up is hardly a shameful idiosyncrasy. top down has a tendency to fail by imposing an inappropriate structure, and missing important details. every discussion i can remember ends up recommending switching directions repeatedly.
i start on the up-stroke myself, always worked for me
i start on the up-stroke myself, always worked for me
The trick to this model is being good enough at figuring out your organization of code that when you get to the easy part of implementation, you don't happen to discover you made a bad assumption earlier.
For me, productivity is 90% about existing in a time and place without blockers. You can have all the right ideas (and even the least productive people I know are full of great ideas) but a single blocker can take your productivity from 100 to 0 indefinitely.
* For programming, this can be knowing a language very well, having smart coworkers who can help with issues, and a plan of how your software will look in the end.
* For getting in shape it can be living near your gym and going to bed at 10:30 instead of 11.
* For writing it can be about turning off message notifications and keeping an outline to prevent writer's block (honestly I'm still working on this one).
I can go on, but this realization has been tremendously useful to me, and lets me exist in "10x productivity mode" many times longer than in the past. While I don't agree with his politics, I strongly believe Scott Adams' ideas around systems vs goals is the right way of thinking of life. You shouldn't expend energy achieving specific goals, you should set your life up so it takes minimal effort to be productive and live a good life. http://blog.dilbert.com/post/102964992706/goals-vs-systems
* For programming, this can be knowing a language very well, having smart coworkers who can help with issues, and a plan of how your software will look in the end.
* For getting in shape it can be living near your gym and going to bed at 10:30 instead of 11.
* For writing it can be about turning off message notifications and keeping an outline to prevent writer's block (honestly I'm still working on this one).
I can go on, but this realization has been tremendously useful to me, and lets me exist in "10x productivity mode" many times longer than in the past. While I don't agree with his politics, I strongly believe Scott Adams' ideas around systems vs goals is the right way of thinking of life. You shouldn't expend energy achieving specific goals, you should set your life up so it takes minimal effort to be productive and live a good life. http://blog.dilbert.com/post/102964992706/goals-vs-systems
Incidentally, have you seen Scott Adams' girlfriend? He's definitely doing something right.
I've struggled with this question lately. However, I think our definition of 'productivity' needs more refinement.
Currently I'm working on a new startup which tries to tackle a rather hard problem (a generic document parser for invoices). Sometimes I'm stuck for days on a particular problem, and I feel like I'm not productive at all, I just stare at a blinking cursor all day. However, in my head I'm constantly working on the problem, and one could say that the moments where I don't write code are actually the most productive, even though it does not feel that way. It's rather counter intuitive.
So I don't think there really isn't a pattern in actual productivity (at least not in my case), although there might be a pattern in how productive you feel.
Currently I'm working on a new startup which tries to tackle a rather hard problem (a generic document parser for invoices). Sometimes I'm stuck for days on a particular problem, and I feel like I'm not productive at all, I just stare at a blinking cursor all day. However, in my head I'm constantly working on the problem, and one could say that the moments where I don't write code are actually the most productive, even though it does not feel that way. It's rather counter intuitive.
So I don't think there really isn't a pattern in actual productivity (at least not in my case), although there might be a pattern in how productive you feel.
I feel you. The time spent discovering a new domain is high-value but also very creatively demanding. Often underestimated by technical folk used to "coding to specification".
That stressful period, full of uncertainty and half-articulated thoughts, as you figure out the basic concepts, how they inter-relate, form goals, make things fall into place conceptually.
Personally, I don't find staring at a blinking cursor helps. During this phase, long walks work the best for me.
That stressful period, full of uncertainty and half-articulated thoughts, as you figure out the basic concepts, how they inter-relate, form goals, make things fall into place conceptually.
Personally, I don't find staring at a blinking cursor helps. During this phase, long walks work the best for me.
Thanks for understanding, it's always nice to hear you're not the only one struggling.
Also great of you to mention walking, I walk a lot during 'work hours'. To outsiders it's so hard to explain that yes, walking is actually part of my process, and yes, walking around in the nearby forest is in some sense my job.
Also great of you to mention walking, I walk a lot during 'work hours'. To outsiders it's so hard to explain that yes, walking is actually part of my process, and yes, walking around in the nearby forest is in some sense my job.
Today I made it out of a local thicket of a problem I've been stuck in for a week and once it worked I was like "well, now that works I can do this, and now _that_ works I can do this" and I got so much done today. But yesterday I went to bed having all but given-up on my path, about to accept defeat after days of failure after failure. Anyway, now it works I can be happy for a day, but tomorrow I know I'll realize this solution is shit and I should really do it another way entirely!
Wouldn't it be cool if invoices had a spec (xml or json) with specific fields that had specific meaning and then we could use something like CSS to make them "pretty" and branded.
This would save from having to parse anything.
To create a bill-of-materials you simply would attach a bunch of invoices from a bunch of vendors. There are jobs out there of people who fetch invoices and then transform them into in-house systems all manually.
This would save from having to parse anything.
To create a bill-of-materials you simply would attach a bunch of invoices from a bunch of vendors. There are jobs out there of people who fetch invoices and then transform them into in-house systems all manually.
This XML spec does exist, it's called UBL. However adoption of this is slow and still quite uncertain. As always, many companies try to make money by creating their own flavour and trying to license it (unfortunate with success) and of course all these competing standards are not compatible with each other.
Since digital bookkeeping is ubiquitous nowadays, one could argue that invoicing is simply about sharing digital transactions between systems, and should not need to be 'pretty' or human readable at all.
I'm trying to bridge this gap, and I know that given enough time, my service will become obsolete.
But I'm goign higly off-topic now ;-)
Since digital bookkeeping is ubiquitous nowadays, one could argue that invoicing is simply about sharing digital transactions between systems, and should not need to be 'pretty' or human readable at all.
I'm trying to bridge this gap, and I know that given enough time, my service will become obsolete.
But I'm goign higly off-topic now ;-)
"my service will become obsolete"
I don't think you need to worry for a long time.
I don't think you need to worry for a long time.
[deleted]
That's the problem, if I was paid for neuron firing and imagination I'd be rich. I'm only paid for some output.
I notice the ebb and flow of productivity as well. Ken uses the term fallow period. I like the reproduction analogy. New ideas, skills, or domain knowledge are like the sperm that finds its place in the egg which I think of as the existing stable but receptive body of knowledge and skills.
There is often excitement at the beginning "I have this new skill!" However, the new skill or knowledge hasn't really integrated with the existing body of knowledge yet. Then often it seems that there needs to be a gestation period. You have the sense that connections are forming and something new is forming that is the combination of the new and the old.
And, then it feels like I am ready to pour it out. Whether it is writing an article, preparing a speech or coding up that feature or application. When that happens it feels like the "Flow." But I understand that the gestation period was just as important if much less visible and seemingly unproductive.
There is often excitement at the beginning "I have this new skill!" However, the new skill or knowledge hasn't really integrated with the existing body of knowledge yet. Then often it seems that there needs to be a gestation period. You have the sense that connections are forming and something new is forming that is the combination of the new and the old.
And, then it feels like I am ready to pour it out. Whether it is writing an article, preparing a speech or coding up that feature or application. When that happens it feels like the "Flow." But I understand that the gestation period was just as important if much less visible and seemingly unproductive.
Just feeling productive doesn't mean you are productive and conversely not feeling productive doesn't mean you are not productive. How do you measure a software developer's productivity? It's very difficult. Reducing the build time might not seem very "productive" but the value it produces over a long time might be higher than any feature or clever code you'd write at this moment.
You can feel like you're producing great code but the proof is in the pudding. It might all need to be thrown away and rewritten.
Value also depends on other factors. Maybe if you spent more time with the junior in the next cube to ensure his piece turns out good the product would actually be successful. You wrote all that good code and the project was cancelled, what was your real productivity?
I go through periods where I don't feel like doing anything and then I go through periods where I have a burning desire to build stuff. Different ideas are in different phases in my mind, some are in long term hibernation, some are ready to execute.
EDIT: I know people who create a lot of code. Even good code. The problem is sometimes we didn't actually need that code at all. Very smart people, wrote a ton of good code, with no leverage. So step #1 in being "productive" is knowing where to put your efforts. I say software engineering is all about leverage, when you have leverage a little bit of code will move the world. With no leverage you can write all the code you want.
You can feel like you're producing great code but the proof is in the pudding. It might all need to be thrown away and rewritten.
Value also depends on other factors. Maybe if you spent more time with the junior in the next cube to ensure his piece turns out good the product would actually be successful. You wrote all that good code and the project was cancelled, what was your real productivity?
I go through periods where I don't feel like doing anything and then I go through periods where I have a burning desire to build stuff. Different ideas are in different phases in my mind, some are in long term hibernation, some are ready to execute.
EDIT: I know people who create a lot of code. Even good code. The problem is sometimes we didn't actually need that code at all. Very smart people, wrote a ton of good code, with no leverage. So step #1 in being "productive" is knowing where to put your efforts. I say software engineering is all about leverage, when you have leverage a little bit of code will move the world. With no leverage you can write all the code you want.
I think there's even another layer that goes with writing code that gets thrown away. Sometimes I end up doing that, but through that I gain knowledge of the problem that I probably couldn't have effectively gained any other way. Should that time spent still be labeled "unproductive"?
Maybe you could have gained the knowledge in other ways? It's hard to tell. It's possible this was absolutely necessary. It's possible a chat with another engineer would have allowed you to get on the right path right away. Who knows.
Lately it's been important for me to think of myself like an elextric motor, two modes:
DRIVE - electrons in, mechanical energy out
REGEN - mechanical energy in, electrons out
Except in your brain it's a little harder to label the i/o... it's something like
CONTROL - cortical activation in, perceptual changes in the world out
SUBMISSION - percepts from the world in, cortical activation out
Just like a vehicle, you stop functioning if you stay too much in one mode or the other. And ironically, I find I achieve maximum control when I cycle back and forth in even measure.
As for long term projects, I find I need to do this cycling at all time scales. Hour, day, week, month, year. It's the cycling back and forth that flushes out nonsense and keeps your finger on a degree of incontrovertible reality.
The trouble is, I often lose track of where I am supposed to be in all these cycles. I am working on software to try to help. Encourage some harmonics.
DRIVE - electrons in, mechanical energy out
REGEN - mechanical energy in, electrons out
Except in your brain it's a little harder to label the i/o... it's something like
CONTROL - cortical activation in, perceptual changes in the world out
SUBMISSION - percepts from the world in, cortical activation out
Just like a vehicle, you stop functioning if you stay too much in one mode or the other. And ironically, I find I achieve maximum control when I cycle back and forth in even measure.
As for long term projects, I find I need to do this cycling at all time scales. Hour, day, week, month, year. It's the cycling back and forth that flushes out nonsense and keeps your finger on a degree of incontrovertible reality.
The trouble is, I often lose track of where I am supposed to be in all these cycles. I am working on software to try to help. Encourage some harmonics.
while in the "Zone", you lose the ability to reflect on the things you are writing - long trip often results in complete bullshit. Which you can discover only upon waking up. Then you get stunned and paralyzed. Sometimes it's a good idea to just throw away the whole thing and start from scratch. I found experimentally that it's good idea to set time limits for your "productivity periods", and keep them rather short.
Everything is easy once you know how. So everything you do breaks down into two phases: learning how, and doing it. During the first phase you feel very unproductive because there are no externally visible results and during the second phase you feel very productive because there are.
I've also been digging into its non-linearity, getting a lot out of this book at the moment: https://www.amazon.com/Peak-Performance-Elevate-Burnout-Scie...
I'm curious about this book. Can you give me one small example of something in this book that helped you?
Sure. It suggests a way to reframe how I think about stress. Not as something harmful to be minimized, but as something that I should intentionally cultivate through activities that are a progression of just-manageable challenges in my chosen endeavor. With each challenge immediately followed by a period of rest that's designed and experienced with just as much intention.
I think that the "feelings of being or not being productive" are often fairly disconnected from the actual levels of productivity.
I do agree that productivity itself exhibits complex patterns, especially with relation to time. These patterns can vary significantly from person to person and over time, though.
I've had inklings of some of own trends for some time now, but it's still pretty fuzzy. I expect to slowly learn more through patient, dispassionate self-observation over the coming years and decades, to the best of my ability. It's tricky stuff, but very interesting.
I do agree that productivity itself exhibits complex patterns, especially with relation to time. These patterns can vary significantly from person to person and over time, though.
I've had inklings of some of own trends for some time now, but it's still pretty fuzzy. I expect to slowly learn more through patient, dispassionate self-observation over the coming years and decades, to the best of my ability. It's tricky stuff, but very interesting.
Productivity is mostly linear for me, judging from non-test lines of code in my commit history. About 36 net lines/hour whether the language is Haskell, C++, or Javascript, or whether it's 3 hours or 14.
There is some variance: Test code increases the ratio, UI code increases the ratio, and debugging lowers the ratio. But it's either wildly off being one of these, or 36-40 lines/hour.
There is some variance: Test code increases the ratio, UI code increases the ratio, and debugging lowers the ratio. But it's either wildly off being one of these, or 36-40 lines/hour.
I'm not a Haskeller, but I frequently hear that Haskell (and FP languages in general) are very terse. Would 36 lines of Haskell not be able do more than 36 lines of C++ of JavaScript?
Anyway, in that regard, LOC is probably a bad metric. Unless the 36 lines of Haskell actually do more, and thus the Haskell/FP language actually allows you to be more productive. That'd be interesting.
Anyway, in that regard, LOC is probably a bad metric. Unless the 36 lines of Haskell actually do more, and thus the Haskell/FP language actually allows you to be more productive. That'd be interesting.
Haskell expressions can indeed be pretty terse (and you get some powerful stuff from fairly standard libraries). But balanced somewhat by type signature lines, data types you might not bother defining in a dynamic language, etc.
For me, Haskell is a little more verbose than Clojure (similarly-powerful expression language, but potentially skipping a bunch of type-related stuff). Maybe comparable to the likes of JavaScript.
APL and Forth-like languages seem to be the place to look for extreme tenseness.
For me, Haskell is a little more verbose than Clojure (similarly-powerful expression language, but potentially skipping a bunch of type-related stuff). Maybe comparable to the likes of JavaScript.
APL and Forth-like languages seem to be the place to look for extreme tenseness.
N lines/hours is a useless metric unless you strictly only do coding to the spec in greenfield projects. E.g. compare using a code generator producing 20k LOC in a millisecond with modifying an undocumented legacy system, where every slight change to the system's incoherent logic might blow up the hole thing.
It's a useless metric in any project. A million line code base can have a lot of value or negative value. If you wrote that at 10 lines/hour your productivity is anything between -X$/hour and +Z$/hour. What does productivity mean in software engineering? Not # of lines of code produced.
I'm saying the code generator would be written at the same net lines/second as the undocumented legacy system. I wouldn't count the 20k LOC as written code. The code generator may be a more efficient project, but the rate at which it's built seems to be the same.
Some would think when we are trying stuff or planning for a project is not getting things done,but the act of planing the right path could get you to point B faster than if you just heads down code and feel productive.
Well, productivity is IS a function of cognitive states. The nonlinearity could be tied to neurotransmitter activity/concentration such as dopamine, or perhaps cortisol level.
I know it's an open door, but when I saw this title and the associated domain I immediately thought "hm, yeah, wow, my productivity is pretty much Perlin noise indeed"
Fun short writing. I like it.
I don't think the two can be separated. Being stuck and being moving are dependent on each other.
A lot of the time I'm frustrated because I haven't quite figured out how the code should look. It takes a long time, a lot of chopping and changing, before you come upon a structure that you feel comfortable building on.
When you do get to that point though, it feels easy. Everything you write makes sense in terms of your new structure, and there aren't so many awkward kludges. The only thing is this period can be brief, because when you're fast, you get to the next problem sooner. You'll find higher and higher levels at which to view your code, and at each level you'll need to do some organising.