Goals Gone Wild: Systematic Side Effects of Over-Prescribing Goal Setting (2009)(hbs.edu)
hbs.edu
Goals Gone Wild: Systematic Side Effects of Over-Prescribing Goal Setting (2009)
http://www.hbs.edu/faculty/Publication%20Files/09-083.pdf
24 comments
This seems like a clear recipe for premature optimization, or "the root of all evil" as Knuth likes to say.
Ending up with an exploding car is also an end result of premature optimization. It means you're focusing on revenue to an extent that is counter-productive.
That means that revenue isn't the overall goal, either. The overall goal is "an ever-flourishing company".
KPI is only one way to try and marshall the resources of a company to move in the right direction, but it seems easy to use wrongly, leading to balkanization of departments, etc. The other way is to use a system like Theory Of Constraints, which encourages a systematic view of the company's overall goals, that is transparent down through the implementation levels, and do regular constraint analysis to identify the (ever-changing) root constraints that block progress towards the overall goals.
Ending up with an exploding car is also an end result of premature optimization. It means you're focusing on revenue to an extent that is counter-productive.
That means that revenue isn't the overall goal, either. The overall goal is "an ever-flourishing company".
KPI is only one way to try and marshall the resources of a company to move in the right direction, but it seems easy to use wrongly, leading to balkanization of departments, etc. The other way is to use a system like Theory Of Constraints, which encourages a systematic view of the company's overall goals, that is transparent down through the implementation levels, and do regular constraint analysis to identify the (ever-changing) root constraints that block progress towards the overall goals.
> "To be successful you set your overall KPI(s), then you forget about it and focus on smaller things that will over time incrementally add to your overall goal."
I don't see how this works in practice. Once you expose KPIs like this, then everything in your org works to increase them. Promotions probably get handed out on this basis. As you say, this leads to exploding cars and bad tools. The challenge (which you don't mention) is how you create an environment that doesn't hold the metrics higher than other things. I'd argue that values-driven orgs probably do this better but I'm not sure.
I don't see how this works in practice. Once you expose KPIs like this, then everything in your org works to increase them. Promotions probably get handed out on this basis. As you say, this leads to exploding cars and bad tools. The challenge (which you don't mention) is how you create an environment that doesn't hold the metrics higher than other things. I'd argue that values-driven orgs probably do this better but I'm not sure.
Yeah that is the challenge. If I knew the right answer that solves it I would certainly share, but I dont.
What I try to do is talk about it over and over, and emphasize for each new initiative we roll out how we dont care about how it will impact the KPIs, we care about how it will impact our overall goals that the KPIs are trying to measure.
That said, #1 resistance point for most initiatives is still "how will affect the KPIs", so we are certainly far away from solving that problem.
Things we've tried to de-emphasize the KPI importance is going away from having an "exceeds" measure, now we are just "meets objective" or "doesnt meet objective", with "doesnt meet" set fairly low so that the people who like to min-max things dont have another clear target to min-max towards, and doing things like moving the focus to team oriented goals rather than individual goals to lower the link between KPIs and rewards, and instead linking individual rewards to general appraisals.
What I try to do is talk about it over and over, and emphasize for each new initiative we roll out how we dont care about how it will impact the KPIs, we care about how it will impact our overall goals that the KPIs are trying to measure.
That said, #1 resistance point for most initiatives is still "how will affect the KPIs", so we are certainly far away from solving that problem.
Things we've tried to de-emphasize the KPI importance is going away from having an "exceeds" measure, now we are just "meets objective" or "doesnt meet objective", with "doesnt meet" set fairly low so that the people who like to min-max things dont have another clear target to min-max towards, and doing things like moving the focus to team oriented goals rather than individual goals to lower the link between KPIs and rewards, and instead linking individual rewards to general appraisals.
For those less initiated in management-speak, KPI is apparently "Key Performance Indicator" - http://en.wikipedia.org/wiki/Key_performance_indicator
Abstract from the paper:
"Goal setting is one of the most replicated and influential paradigms in the management literature. Hundreds of studies conducted in numerous countries and contexts have consistently demonstrated that setting specific, challenging goals can powerfully drive behavior and boost performance. Advocates of goal setting have had a substantial impact on research, management education, and management practice. In this article, we argue that the beneficial effects of goal setting have been overstated and that systematic harm caused by goal setting has been largely ignored. We identify specific side effects associated with goal setting, including a narrow focus that neglects non-goal areas, a rise in unethical behavior, distorted risk preferences, corrosion of organizational culture, and reduced intrinsic motivation. Rather than dispensing goal setting as a benign, over-the-counter treatment for motivation, managers and scholars need to conceptualize goal setting as a prescription-strength medication that requires careful dosing, consideration of harmful side effects, and close supervision. We offer a warning label to accompany the practice of setting goals."
"Goal setting is one of the most replicated and influential paradigms in the management literature. Hundreds of studies conducted in numerous countries and contexts have consistently demonstrated that setting specific, challenging goals can powerfully drive behavior and boost performance. Advocates of goal setting have had a substantial impact on research, management education, and management practice. In this article, we argue that the beneficial effects of goal setting have been overstated and that systematic harm caused by goal setting has been largely ignored. We identify specific side effects associated with goal setting, including a narrow focus that neglects non-goal areas, a rise in unethical behavior, distorted risk preferences, corrosion of organizational culture, and reduced intrinsic motivation. Rather than dispensing goal setting as a benign, over-the-counter treatment for motivation, managers and scholars need to conceptualize goal setting as a prescription-strength medication that requires careful dosing, consideration of harmful side effects, and close supervision. We offer a warning label to accompany the practice of setting goals."
I've witnessed this first hand - we recently started rolling out measured goals in our warehouses. This involved staff getting set daily goal counts for certain tasks (products shipped, products put on shelves etc.). As soon as we rolled out metrics everything that wasn't measured became a low priority. For example, we were not measuring cycle counts (stock take) so immediately the number of cycle counts plummeted, which increased our out-of-stock errors. We had instances of staff hoarding incoming shipments from suppliers at their workstations in order to get higher rankings.
Overall the warehouses went from being generally efficient to extreme performers on measured metrics. I think metrics can be clearly powerful, but you have to be very careful about what metrics you choose to implement.
Overall the warehouses went from being generally efficient to extreme performers on measured metrics. I think metrics can be clearly powerful, but you have to be very careful about what metrics you choose to implement.
I really wish this was taught more in business school: that metrics/productivity is subject to the Observer Effect (http://en.wikipedia.org/wiki/Observer_effect_%28physics%29) too: that the act of observing for metrics will change the underlaying system.
If underperforming on your metrics will cause some pain, humans in general want to avoid pain, and you ARE employing smart people who are paid to solve problems and do analysis ... well, duh some metrics gaming will happen: people avoiding pain.
Probably this will have an opposite affect than what the measurer wanted: people hording things/information/output specifically to improve their metrics, people working around the metrics (outside the managed systems), or essentially spamming the system ("oh you found a bug in my you're reviewing? Can you file a new bug on this?... so my closed bug count for the week goes up...")
If underperforming on your metrics will cause some pain, humans in general want to avoid pain, and you ARE employing smart people who are paid to solve problems and do analysis ... well, duh some metrics gaming will happen: people avoiding pain.
Probably this will have an opposite affect than what the measurer wanted: people hording things/information/output specifically to improve their metrics, people working around the metrics (outside the managed systems), or essentially spamming the system ("oh you found a bug in my you're reviewing? Can you file a new bug on this?... so my closed bug count for the week goes up...")
A long time ago, in western Europe, kids wouldn't go to school if they were sick. To check if they were sick the mother would check their temperature with an alcohol thermometer. Of course the kids understood the goal and used the night lamp to get the appropriate temperature displayed to skip school.
When talking about metric in corporate setting I like to remind everybody that I have a strong suspicion that the guy who invented the thermometer also discreetly invented the night lamp.
(most of the context is disappearing: blinds that force you to use the night lamp in the morning, alcohol thermometers and incandescent light bulbs)
When talking about metric in corporate setting I like to remind everybody that I have a strong suspicion that the guy who invented the thermometer also discreetly invented the night lamp.
(most of the context is disappearing: blinds that force you to use the night lamp in the morning, alcohol thermometers and incandescent light bulbs)
Eating raw potato worked too. Eastern Europe, not so long time ago. :)
My memory of this involves dunking the thermometer in the cup of tea.
I did the same. The top of the thermometer exploded as the mercury inside got too large. Apparently, the thermometer was not designed to measure freshly made tea.
I work as a contractor for a company where employees must set a couple of goals yearly and meet them. Set them too low, you're considered lazy, fail to meet one and you're considered unreliable.
There's also a mandatory quota of trainings, which employees must elaborate and then train their colleagues, and "charitable work" they must participate.
I don't want to be hired.
There's also a mandatory quota of trainings, which employees must elaborate and then train their colleagues, and "charitable work" they must participate.
I don't want to be hired.
"Set them too low, you're considered lazy, fail to meet one and you're considered unreliable."
Ah, yes. "You must be able to handle ambiguity, because we can't."
Ah, yes. "You must be able to handle ambiguity, because we can't."
A glass of wine per day is ok. Maybe even good. Two six-packs a day isn't. Anything is ok in moderation, and with full knowledge of both positive and negative effects. I have long-term objectives for sure, but avoid setting long-term SMART objectives for myself, because life happens. At work during the annual review I set SMART objectives as required, but completely ignore if not forget them during the rest of the year.
Short term is another story. Every evening I decide exactly what I want to accomplish during the next 24 hours, always looking at how what I'm planning for the day aligns with my longer term objectives. That's been crazy useful to me.
Amusingly, that sentiment has a middle ground which I apply to software development projects - I live and die by a project's vision/scope, which is really an objective for a project.
Short term is another story. Every evening I decide exactly what I want to accomplish during the next 24 hours, always looking at how what I'm planning for the day aligns with my longer term objectives. That's been crazy useful to me.
Amusingly, that sentiment has a middle ground which I apply to software development projects - I live and die by a project's vision/scope, which is really an objective for a project.
Agreed on all points, and I'd like to add some numbers that worked for me - keep about 5-10 long-term objectives. Make one of them taking care of your health (exercise, nutrition), one taking care of your mental health (mostly work/rest/socialize balance), and invest in your knowledge/skills. Currently, I have 5 more (finish project #1, #2, #3, make company profitable, move out of country).
Additional point I'd like to make is - know thyself. You simply cannot set goals on some wishful thinking and other randomnesses. While glass of wine per day is ok, you might better be off without any, or you might need a bottle of wine a day to manage your cholesterol level (stretching the metaphore).
Also, props for planning out the next day in evening - makes sleep and the start of next day so much more pleasurable experience.
Additional point I'd like to make is - know thyself. You simply cannot set goals on some wishful thinking and other randomnesses. While glass of wine per day is ok, you might better be off without any, or you might need a bottle of wine a day to manage your cholesterol level (stretching the metaphore).
Also, props for planning out the next day in evening - makes sleep and the start of next day so much more pleasurable experience.
Part of the problem here is the problem of aggregation.
We measure things in the aggregate, but we influence them in the minuscule.
So I might measure the fact that a certain fast food restaurant makes french fries slower than another one, but there could be a thousand reasons why this is so. Simply setting a goal of increasing the number of french fries delivered doesn't actually make it happen. Instead, it ends up perversely impacting all the other areas of the restaurant.
We look at things in big, fuzzy ways, so we naturally think we can influence them in the same way. We measure in the aggregate, then find some correlations in the aggregate, declare causality, then set goals. Doesn't work like that. This is the way things are commonly done, and there are multiple logical errors here. [Add in long discussion about the implications of this on public policy-making]
We measure things in the aggregate, but we influence them in the minuscule.
So I might measure the fact that a certain fast food restaurant makes french fries slower than another one, but there could be a thousand reasons why this is so. Simply setting a goal of increasing the number of french fries delivered doesn't actually make it happen. Instead, it ends up perversely impacting all the other areas of the restaurant.
We look at things in big, fuzzy ways, so we naturally think we can influence them in the same way. We measure in the aggregate, then find some correlations in the aggregate, declare causality, then set goals. Doesn't work like that. This is the way things are commonly done, and there are multiple logical errors here. [Add in long discussion about the implications of this on public policy-making]
There are many types of ambition.
Goal-focused ambition is only one type, it may be the most recognizable, maybe due to the influence of goal-seeking sports competitions.
Goals are great for some games, some of which are competitive. Not everything is a game.
Sometimes a team benefits from a goal more than an individual.
What if you have a goal that is not recognized or appreciated?
What if you've already reached your goals?
What if you have stronger ambition by nature than the goal-seeking type?
What if your goal was to perform without a specific goal while still outperforming those who focused on it?
What if you recognized the factors identified in the original PDF decades ago and groomed yourself to be able to sometimes engage in more effective goal-seeking than the pure devotees, while also outperforming them toward their own goals while yourself being unfocused, whenever you wanted to according to the situation?
Goal-focused ambition is only one type, it may be the most recognizable, maybe due to the influence of goal-seeking sports competitions.
Goals are great for some games, some of which are competitive. Not everything is a game.
Sometimes a team benefits from a goal more than an individual.
What if you have a goal that is not recognized or appreciated?
What if you've already reached your goals?
What if you have stronger ambition by nature than the goal-seeking type?
What if your goal was to perform without a specific goal while still outperforming those who focused on it?
What if you recognized the factors identified in the original PDF decades ago and groomed yourself to be able to sometimes engage in more effective goal-seeking than the pure devotees, while also outperforming them toward their own goals while yourself being unfocused, whenever you wanted to according to the situation?
"Given that small actions within an organization can have broad implications for organizational culture (Fleming & Zyglidopoulos, 2008), we postulate that aggressive goal setting within an organization will foster an organizational climate ripe for unethical behavior. That is, not only does goal setting
directly motivate unethical behavior, but its introduction may also motivate unethical behavior indirectly by subtly altering an organization’s culture. In sum, although many factors contribute to unethical behavior, the point cannot be overstated: goal setting motivates unethical behavior."
What a rubbish article. Without goals, how would anyone know what needs to be done? How would you know what is expected of you? If it is a generic company, the main goals are to make sure the company stays around and to make a profit. You then create sub-goals to achieve the main goals and repeat this until everyone knows what needs to be done. If the sub-goals do not agree with the main goals you get the mess which the article describes. This is caused by WRONG goals.
While my initial reaction is similar to yours, keep in mind that what's being proposed by the article is the concept that no one sets goals well enough that they should be used the way they are right now. This is a premise I don't agree with, mostly because I think their data is really shallow, but the point is that they are proposing a reality in which you cannot find a decent dataset of people setting goals well within their company.
Obviously, the goal of playing is to make money, and almost equally obviously, focusing on how much money you made is actively harmful if done for sample sizes lower than 100k hands or so.
The solution to improvement through quantitative analysis in that environment is to find key drivers that you believe impact your overall goal, analyse trends you see in those drivers, and based on those find a few areas that you want to qualitatively drill into to figure out changes you can make. I.e. - "I open 20% of the time from early position and 25% from late position - if I compare to other successful players that is too low a difference, so lets see if I can find some situations where I may be making bad decisions and plug those holes".
That is what we want to do with goal setting in business as well. Yes we have some high level KPIs that we want to improve, but actively working "to improve your revenue" is harmful and will no doubt lead to selling cars that explode etc. To be successful you set your overall KPI(s), then you forget about it and focus on smaller things that will over time incrementally add to your overall goal.
If you have managers and employees who cant make that separation or want to take short cuts to directly affect the overall goal without focusing on the smaller bits, or comparing to some kind of "best known" practise, then you end up with exploding cars, tools that has all features and no usability etc.