Management by metrics leads us astray(jakobgreenfeld.com)
jakobgreenfeld.com
Management by metrics leads us astray
https://jakobgreenfeld.com/metrics
249 comments
Hi, apart from Hayek (and maybe Ursula Le Guin, and A Hamilton), has anyone got any good pointers on theories for when you favour local over global knowledge or vice versa. Either political theories, or game-theoretic. Transaction cost economics is the closest thing I've seen, and maybe some of Stuart Kauffman's stuff on the adjacent possible and the CEO's job being to 'recut' the company into different divisions. But I've never seen anything along the lines of - if I have a changing fitness landscape L and a set of communicating agents A with different sets of knowledge, and they can use their knowledge to extract value from the local landscape, or they can make tools to increase the value of that knowledge or they can communicate or they can listen, then in what situations should you take different actions. The extremes are easy ( local changing landscape, high comms costs, then don't listen to anyone else; static similar landscape, cheap comms, listen to the cleverest agent); Maybe there is some biology stuff with ants or some agent-based modelling work ?
I can recommend two authors: Nassim Taleb and James C Scott. In particular their books "Antifragile" and "Seeing Like A State" respectively.
Controversy over Taleb is no secret, but I'll say it every time. Everything I've seen by Taleb seems to be him posing an idea that is obviously flawed, claiming that it is mainstream among widespread experts (when its not), and then taking credit for saying its incorrect, whilst making the reader feel smart because they too see why its obviously wrong.
I'm not sure this is true.
Sometimes he says things that are, well, nonsense (e.g. a lot of his anti-GMO posturing), and other times he's right on the money (e.g. criticizing Black-Scholes long before the '08 crisis). In all cases, he insults his detractors as "bullshitters" when it is only warranted for some.
Sometimes he says things that are, well, nonsense (e.g. a lot of his anti-GMO posturing), and other times he's right on the money (e.g. criticizing Black-Scholes long before the '08 crisis). In all cases, he insults his detractors as "bullshitters" when it is only warranted for some.
Both great books!
Lean manufacturing or Toyota production system is pretty relevant. The concept of sub-optimizing metrics seemed especially topical to the article.
Demming’s Out of the Crisis talks about reducing variation to improve efficiency and how difficult that is.
Francis Spufford’s Red Plenty is a great “nonfiction novel” about the problems of (and conceptual/mathematical limits to) central planning in the Soviet Union.
Kind of unrelated but when you mentioned Soviet shoe factories, this scene from HBO's Chernobyl takes on a whole new dimension:
https://www.youtube.com/watch?v=idb_qsAAe1c
"I'm a nuclear physicist. Before you were deputy secretary, you worked in a shoe factory."
"Yes, I worked in a shoe factory. And now I'm in charge. To the workers of the world."
https://www.youtube.com/watch?v=idb_qsAAe1c
"I'm a nuclear physicist. Before you were deputy secretary, you worked in a shoe factory."
"Yes, I worked in a shoe factory. And now I'm in charge. To the workers of the world."
When writing my post, I remembered the miner's joke from the series:
> What's as big as a house, burns 20 liters of fuel an hour, puts out a huge amount of smoke and noise, and cuts an apple into three pieces?
> A Soviet machine for cutting apples into four pieces.
> What's as big as a house, burns 20 liters of fuel an hour, puts out a huge amount of smoke and noise, and cuts an apple into three pieces?
> A Soviet machine for cutting apples into four pieces.
The USSR had the same issues that large companies do because central planning was modeled on a corporate model.
Internally, corporations function like centrally planned economies.
Internally, corporations function like centrally planned economies.
> Internally, corporations function like centrally planned economies.
Well, they try to. But unlike Governments, they don't have access to Departments of economists or planners that can spend all their time looking at the data and deciding the optimal plan. The decisions corporations take is most often based on the whims of its decision makers (senior managers/HOD/VP of X) who will cherry-pick metrics that suits their perspective.
Well, they try to. But unlike Governments, they don't have access to Departments of economists or planners that can spend all their time looking at the data and deciding the optimal plan. The decisions corporations take is most often based on the whims of its decision makers (senior managers/HOD/VP of X) who will cherry-pick metrics that suits their perspective.
>> who will cherry-pick metrics that suits their perspective.
Your belief is that governments never do this, that government decision makers do not make choices off the their own personal whims??
Your belief is that governments never do this, that government decision makers do not make choices off the their own personal whims??
You seem to think that teams of economists don't cherry pick metrics that suit their perspective.
This is a strange argument.
Government departments dedicated to planning are often required to publish their guidance/research publicly making it difficult to get away with cherry-picking metrics.
Having a team of professionals who understand the basics of our economic system tackle the problem seems to have a higher probability of success as compared with business leaders who often lack formal training in statistical methods.
There are more arguments but this should be enough to prove this false equivalency.
Government departments dedicated to planning are often required to publish their guidance/research publicly making it difficult to get away with cherry-picking metrics.
Having a team of professionals who understand the basics of our economic system tackle the problem seems to have a higher probability of success as compared with business leaders who often lack formal training in statistical methods.
There are more arguments but this should be enough to prove this false equivalency.
I see a few problems with this framing: First, professionals in government can and do often get into turf wars about their research. I think anyone who has ever worked in government knows this is the case. If you doubt me, ask government researchers to publish their hypotheses ahead of time before running a study, and see how many government researchers are willing to do this. (I'd be surprised if you got an email or even a tweet back.)
Second, decision making is always inherently political; it cannot be assumed away by throwing "professionals" at the problem. This is a consequence of "value" being entirely subjective. When economics was "political economy" (or even a field within "moral philosophy"), the field understood this. But that arrangement didn't give the professionals and academics enough status or prestige, so American economists in the late 19th century who had studied in Bismarck-era Germany rebranded themselves as "economists" who were practicing a form of science not too different from, say, physics.
Third, business leaders have to make a profit and maintain positive cash flows in order to sustain themselves. If they don't, the business dies. This is an extremely hard constraint that disciplines any business leader regardless of statistical literacy. States, on the other hand, can always subsist on the use of force to extract revenue. Central planners can publish metrics till they're blue in the face, but what constraints are they under to make the metrics any good?
Second, decision making is always inherently political; it cannot be assumed away by throwing "professionals" at the problem. This is a consequence of "value" being entirely subjective. When economics was "political economy" (or even a field within "moral philosophy"), the field understood this. But that arrangement didn't give the professionals and academics enough status or prestige, so American economists in the late 19th century who had studied in Bismarck-era Germany rebranded themselves as "economists" who were practicing a form of science not too different from, say, physics.
Third, business leaders have to make a profit and maintain positive cash flows in order to sustain themselves. If they don't, the business dies. This is an extremely hard constraint that disciplines any business leader regardless of statistical literacy. States, on the other hand, can always subsist on the use of force to extract revenue. Central planners can publish metrics till they're blue in the face, but what constraints are they under to make the metrics any good?
The hope is that Darwin will select for the corporations that do this better.
Given how few corporations last any significant length of time, it works until it doesn’t.
As someone who grew up in late Communist Czechoslovakia, the regime was absolutely obsessed with production of impressive statistics, especially those of heavy industry.
How much steel was produced yearly, how much coal was produced yearly, and how the old bourgeois Czechoslovakia (1918-1939) was so much worse off in comparison.
What was NOT trumpeted out loud:
* we were seriously behind in production of many consumer goods and in their quality, * some parts of our industry could not compete with Western products on the account of being obsolete - especially electronics.
How much steel was produced yearly, how much coal was produced yearly, and how the old bourgeois Czechoslovakia (1918-1939) was so much worse off in comparison.
What was NOT trumpeted out loud:
* we were seriously behind in production of many consumer goods and in their quality, * some parts of our industry could not compete with Western products on the account of being obsolete - especially electronics.
Having never lived in a socialist country, but having seen the effects up close and having read about them extensively, my model of Marxist-Leninist states is that they are basically giant corporations that produce everything and employ everyone within the boundaries of a state. Imagine Walmart, but they produce everything, employ everyone, "Loss Prevention" runs a corporation-wide spying ring (with incentives for turning in your fellow employees), HR has show trials and prisons/camps for anyone who might be causing low worker morale. Management releases quarterly numbers that are extremely rosy, mainly for the benefit of the board (the Politburo or maybe Central Committee) and the shareholders (the Party), though this is mainly to enforce which talking points to discuss; none of them can say anything but disparaging things about the competition (bourgeois capitalist-imperialist states), nor acknowledge any progress they may have achieved. Above all, management wants everyone to know that they are actually a giant family.
I'm not sure how far off this is, but socialist propaganda always reminds me of corporate BS I've seen over the course of my career. At least with a corporation, I can leave.
I'm not sure how far off this is, but socialist propaganda always reminds me of corporate BS I've seen over the course of my career. At least with a corporation, I can leave.
Your model is nice, but it is missing one important step: At the beginning of the experiment, all people who were not at the bottom of the company pyramid are shot, and their roles are filled by randomly selected remaining employees. So the company starts with nobody in the management having any idea about anything, and it takes a few decades of trial and error to regain some knowledge about how to run things. In the meanwhile, many employees starve to death.
At the end, Wallmart finally regains its former functionality. The management proudly opens the windows that have been locked for decades. On the outside, they see drones flying everywhere, a personal teleport at every street corner, and rockets regularly flying to Mars. In despair, the entire project is abandoned.
At the end, Wallmart finally regains its former functionality. The management proudly opens the windows that have been locked for decades. On the outside, they see drones flying everywhere, a personal teleport at every street corner, and rockets regularly flying to Mars. In despair, the entire project is abandoned.
"Frontline" ran an episode some years back where they investigated some dental clinics that did astonishing numbers of unnecessary major dental operations.
It turns out that the government had a program that paid for these operations when performed on poor people. So the clinics would convince their customers that they needed major dental work, then would do the work and bill the government. The clinics were totally specialized just for this.
"Frontline" pinned the blame on the clinics, sort of "how could the do that". Of course, they're right. But blaming the clinics doesn't work, because the actual problem was the government set things up so that the rise of clinics to exploit the rules was legal and inevitable.
You're now faced with the problem of crafting a bureaucratic rule to define a "necessary" major dental operation. Of course, it's impossible, and the same problem as "management by metrics".
It turns out that the government had a program that paid for these operations when performed on poor people. So the clinics would convince their customers that they needed major dental work, then would do the work and bill the government. The clinics were totally specialized just for this.
"Frontline" pinned the blame on the clinics, sort of "how could the do that". Of course, they're right. But blaming the clinics doesn't work, because the actual problem was the government set things up so that the rise of clinics to exploit the rules was legal and inevitable.
You're now faced with the problem of crafting a bureaucratic rule to define a "necessary" major dental operation. Of course, it's impossible, and the same problem as "management by metrics".
"Blaming the clinics doesn't work" - I trying to wrap my head around to put myself in your frame of mind to shift the blame from who is to blame to the government. If those clinics recommend and perform unnecessary procedures on people that can afford them (and do not benefit from govt. aid), does "blaming the clinics not work", because the actual problem is people being able to afford it? or would you go one step ahead and then say that the government is still to blame for allowing the possibility of wealth accumulation by individuals that results in unnecessary spending? or is the government still to blame because of not further regulating and auditing dental clinics? or is the government to blame because of allowing the mere existence of private dental clinics?
I guess that the government did not "set things up so that the rise of clinics to exploit the rules was legal and inevitable". I am going to guess that individuals, knowing that that was not the intention of the government aid, decided they could get around it to profit.
edit: typos
I guess that the government did not "set things up so that the rise of clinics to exploit the rules was legal and inevitable". I am going to guess that individuals, knowing that that was not the intention of the government aid, decided they could get around it to profit.
edit: typos
If you have a plan that relies on everyone in a nation of 300 million behaving ethically and honestly when there is a big incentive not to, you have a bad plan.
The moral failing is on the clinics, sure, but if you want to fix the problem, you have to look at the government.
The moral failing is on the clinics, sure, but if you want to fix the problem, you have to look at the government.
Dentists take the Hippocratic Oath just like doctors. First and foremost, this wasn't just a moral failing, this was a major professional violation, regardless of any government incentives. They put patients in unnecessary risk of complications and likely caused harm in some cases, in direct violation of the one of the most important oaths in any profession. It is the job of healthcare professionals to cater to the health of their patients, not their billing departments.
You can't write all laws like we're living in a Mad Max dystopia - some laws will need to assume that other laws are functioning correctly in order to be practically implementable as a law - just like software has dependencies that must uphold some invariants for the interface to work. The issue isn't subsidizing healthcare, it's unethical professionals abusing our trust.
You can't write all laws like we're living in a Mad Max dystopia - some laws will need to assume that other laws are functioning correctly in order to be practically implementable as a law - just like software has dependencies that must uphold some invariants for the interface to work. The issue isn't subsidizing healthcare, it's unethical professionals abusing our trust.
"Dentists take the Hippocratic Oath just like doctors. First and foremost, this wasn't just a moral failing, this was a major professional violation, regardless of any government incentives. They put patients in unnecessary risk of complications and likely caused harm in some cases, in direct violation of the one of the most important oaths in any profession."
Generally true, but the judgment call on "is this patient better off by being left alone" vs. "is this patient better off by having a surgery" is not easy, especially in our culture which worships heroic medicine. Patients often demand that something be done because they are primed so from the popular media they consume.
This can be even seen in the end-of-life care. Too many families demand that their dying grandma is given expensive and invasive care to live three weeks longer, even though its not just pointless, but outright cruel.
Generally true, but the judgment call on "is this patient better off by being left alone" vs. "is this patient better off by having a surgery" is not easy, especially in our culture which worships heroic medicine. Patients often demand that something be done because they are primed so from the popular media they consume.
This can be even seen in the end-of-life care. Too many families demand that their dying grandma is given expensive and invasive care to live three weeks longer, even though its not just pointless, but outright cruel.
> Generally true, but the judgment call on "is this patient better off by being left alone" vs. "is this patient better off by having a surgery" is not easy, especially in our culture which worships heroic medicine. Patients often demand that something be done because they are primed so from the popular media they consume.
That's why it's a problem with professional ethics in healthcare first and foremost. Nobody in the bureaucratic chain is qualified to make or judge medical decisions and them trying would only be a monumental disaster because they don't even meet the patient and they're probably not medical professionals themselves. Hell, the patients and their family are rarely qualified to make the decisions - just look at what doctors chose for end of life care versus what their patients end up doing.
The system doesn't work if we don't police the professionals making the decisions at the point of care. The trick is not overdoing it like we did with opiates, scaring doctors from prescribing them even for patients genuinely in need.
That's why it's a problem with professional ethics in healthcare first and foremost. Nobody in the bureaucratic chain is qualified to make or judge medical decisions and them trying would only be a monumental disaster because they don't even meet the patient and they're probably not medical professionals themselves. Hell, the patients and their family are rarely qualified to make the decisions - just look at what doctors chose for end of life care versus what their patients end up doing.
The system doesn't work if we don't police the professionals making the decisions at the point of care. The trick is not overdoing it like we did with opiates, scaring doctors from prescribing them even for patients genuinely in need.
Isn't the answer to this better accountability? Incentives without accountability usually lead to this type of outcome.
Define accountability. Now you have a cost benefit ratio. If there is personal liability who can be blamed so you get the benefit and they get the detriment.
In this specific case, it would be an independent audit of the procedures to determine whether they were actually necessary or not.
it's not rocket science. Poor people don't have the money required to go to the dentist and properly take care of your teeth. Since having a healthy populace is a good thing, we say "okay, we're all going to pitch in so everyone can be as healthy as possible."
If people start to take advantage of that for personal gain (which we know there are always people who are going to do that), we need to have a plan in place for dealing with those assholes.
The presence of assholes in society shouldn't mean we don't even try to take care of each other. It should mean that we have plans for recognizing when someone is an asshole and taking measures to change that behavior.
it's not rocket science. Poor people don't have the money required to go to the dentist and properly take care of your teeth. Since having a healthy populace is a good thing, we say "okay, we're all going to pitch in so everyone can be as healthy as possible."
If people start to take advantage of that for personal gain (which we know there are always people who are going to do that), we need to have a plan in place for dealing with those assholes.
The presence of assholes in society shouldn't mean we don't even try to take care of each other. It should mean that we have plans for recognizing when someone is an asshole and taking measures to change that behavior.
Now you've created a new problem of bureaucrats that are just as susceptible to incompetence, incentives, and malfeasance as the dentists.
This is going to be the problem with every system of mandatory funding. Doesn't matter if it's housing, dentistry, food stamps, etc. People were abusing the disability program, so we made a bunch of hurdles to get funding. Now there's companies who train you how to get over those hurdles: what to write on the forms, what to say to your doctor, etc. And guess what? The companies that train you what to say to get disability then get to bill the government for this new service they provided you. I'm sure it's easy to imagine their incentives.
This is going to be the problem with every system of mandatory funding. Doesn't matter if it's housing, dentistry, food stamps, etc. People were abusing the disability program, so we made a bunch of hurdles to get funding. Now there's companies who train you how to get over those hurdles: what to write on the forms, what to say to your doctor, etc. And guess what? The companies that train you what to say to get disability then get to bill the government for this new service they provided you. I'm sure it's easy to imagine their incentives.
And yet there are ways we can address those issues too. I don't get why we gotta stop trying to take care of each other simply because there are assholes who try to take advantage of things.
>I don't get why we gotta stop trying to take care of each other
How would ending food stamps stop you from taking care of other people?
How would ending food stamps stop you from taking care of other people?
Removing a massive mechanism for being able to address a need that most people feel shame for and explicitly try to hide due to social pressure?
If I'm acting on my own, my reach is very limited. By using the government, we have the potential to make a much larger impact, and in this case, create the expectation that no one should go hungry in this country. We're not there yet, mainly because there are many people around here who don't believe that expectation is valid. I'll fight them for the rest of my life.
If I'm acting on my own, my reach is very limited. By using the government, we have the potential to make a much larger impact, and in this case, create the expectation that no one should go hungry in this country. We're not there yet, mainly because there are many people around here who don't believe that expectation is valid. I'll fight them for the rest of my life.
There is a specific term relevant to this, which is applicable to many problems of misaligned risk/incentive, governmental and otherwise: "Moral Hazard"
Seems like existing whistle blower incentives could be beefed up and better publicized.
A new "moral heroes" national holiday celebrating those who uncover fraud. More resources dedicated to investigating claims, more leeway to fine corporations for bad faith activity in order to increase the incentive to blow the whistle.
A new "moral heroes" national holiday celebrating those who uncover fraud. More resources dedicated to investigating claims, more leeway to fine corporations for bad faith activity in order to increase the incentive to blow the whistle.
I think you are implying the following when you say "you have to look at the government": "if you want to fix the problem of people/clinics/etc. abusing (dodging/fraudulently take advantage of) government programs/policies you have to ..." ----- And the "..." is not filled in. I guess I could fill it in with: "stop implementing policies/programs if you are not at the same time investing as much as it is necessary to guarantee nobody gets around the law" - that is, do not trust, add more inefficiency (clinics already need to attest the procedure is necessary) and cost.
Dentists and doctors are not all of one mind about treatment effectiveness or necessity, so the tails of the bell curve matter. There may very well be some small percentage of dentists who believe that procedure X is beneficial in almost all cases. They might be quacks, but most quacks I've met believe what they are selling (even if they are wrong.)
If there also is a government program that reimburses for X in almost all cases, then you could see the emergence of clinics that "exploit" the rule even if everyone involved believes they are behaving ethically.
There are feedback loops that then exacerbate the problem- for example a successful clinic might bring on additional dentists, who despite misgivings will be influenced by existing practice at the clinic and begin self justifying the procedures to conform to the senior partner. After all, they clearly know what they are doing, right?
While I have no idea if that's what happened here, it is well with in the realm of possibility. The government does bear some responsibility to ensure that thier programs and rules provide sufficient oversight and feedback to keep fringey participants within the bounds of what the government considers acceptable.
If there also is a government program that reimburses for X in almost all cases, then you could see the emergence of clinics that "exploit" the rule even if everyone involved believes they are behaving ethically.
There are feedback loops that then exacerbate the problem- for example a successful clinic might bring on additional dentists, who despite misgivings will be influenced by existing practice at the clinic and begin self justifying the procedures to conform to the senior partner. After all, they clearly know what they are doing, right?
While I have no idea if that's what happened here, it is well with in the realm of possibility. The government does bear some responsibility to ensure that thier programs and rules provide sufficient oversight and feedback to keep fringey participants within the bounds of what the government considers acceptable.
I understand your arguments, which in my view, "shift the blame", and in your view, "charge with at least some responsibility" to the government. The problem I see is that this sort of arguments, if accepted as OK/inevitable and universalized lead to a dilution of responsibility that would make no one fully accountable for any intentional wrongdoing: for example, if you pay for your car to be repaired at your regular car repair shop, but the mechanic knowingly used parts that were not certified or due to time pressure skipped standard checks, and you let someone drive your car a year later, and there is an accident that was caused as a consequence of that sloppy repair-work, do you share responsibility for your oversight? - You might argue you do; well, I would argue you don't.
The government is not an individual- appropriate morality and responsibility at scale are different then as an individual. Additionally, if insurance / the government agrees to pay for a procedure that serves as a form of confirmation for the end user - after all, the department of health and human services approved the procedure!
A better example would be if a Sr Director of a rental car company shifted sourcing of thier repair contractor. If the new repair contractor consistently did way more repairs then the rental company needed, should the director of fleet maintenance be held responsible? Absolutely.
A better example would be if a Sr Director of a rental car company shifted sourcing of thier repair contractor. If the new repair contractor consistently did way more repairs then the rental company needed, should the director of fleet maintenance be held responsible? Absolutely.
The moral blame rests fully on the clinic, but the responsibility to fix the system rests fully on the government (in this case). It's really no different than a production line with a quality problem. You can yell and scream about the morality of the workers all you want, but you can't expect them to improve if you don't fix the system and incentives that cause the problems.
the reason the analogy doesn’t hold up is there isn’t a system there — that’s a set of actually individual choices. When you see systematic behavior, you need to look for the system to fix it. The problem with blame is it doesn’t get you far enough, even if it feels good or right. Finding and fixing incentives is the only way to systematically enact change. Whether it feels good or not.
btw, I think it is important to place the blame correctly and take whatever action is possible on the perpetrators to deter such behavior and at the same time patch systemic loopholes that are identified. They are not incompatible actions. If anything, they are complementary.
The same dynamic exists for private insurance companies. The existence of incentives for illegal activity doesn't shift the blame off the criminal.
We don't say that an ATM mugging victim bears some responsibility just because a criminal knows that such people will almost always pay out.
We don't say that an ATM mugging victim bears some responsibility just because a criminal knows that such people will almost always pay out.
blame/responsibility is a red herring here. mugging is already illegal and, if caught, the perpetrator faces imprisonment. the salient question to ask here is what changes we can make to reduce the unwanted behavior.
possible actions:
politely ask the general public not to mug one another
increase penalties or rate of apprehension for perpetrators
encourage people to minimize their ATM visits, not walk around with debit cards, etc.
give people money so they don't feel the need to mug other people
possible actions:
politely ask the general public not to mug one another
increase penalties or rate of apprehension for perpetrators
encourage people to minimize their ATM visits, not walk around with debit cards, etc.
give people money so they don't feel the need to mug other people
You're right, "blame" is too loaded of a word, and also inadequate. However, in order to identify the actions to take, you still need something like a full systems analysis of weak points. Your list is a pretty good start on representing those.
That requires setting up objective bureaucratic rules, which are the same problem as "Management by metrics leads us astray."
I think we can safely blame both owners and employees of the clinics and the people in government who devised and implemented the program.
We can blame the owners of the clinics because we have a right to expect people to to make morally correct decisions. We will never invent the perfect system of laws to incentivize everyone's good behavior. At the end of the day, society only functions when the majority of people are at least somewhat selflessly contributing to it.
We can blame those who devised and implement the program because we can expect them to do more auditing as well and ensuring the program isn't gamed. People in government are responsible for spending taxpayer money wisely.
We can blame the owners of the clinics because we have a right to expect people to to make morally correct decisions. We will never invent the perfect system of laws to incentivize everyone's good behavior. At the end of the day, society only functions when the majority of people are at least somewhat selflessly contributing to it.
We can blame those who devised and implement the program because we can expect them to do more auditing as well and ensuring the program isn't gamed. People in government are responsible for spending taxpayer money wisely.
The trouble is that blame doesn’t address the problem either. We want the government to make wiser, less gameable policy. We want businesses to act ethically. How do we acheive those? It’s not blame.
I think the answer is incentives. The government had the incentive to “help the poor” - so they did. The clinics had the incentive to “fix teeth for people in need” - so they did.
Just like in software development, so much rests on properly defining the desired outcome.
I think the answer is incentives. The government had the incentive to “help the poor” - so they did. The clinics had the incentive to “fix teeth for people in need” - so they did.
Just like in software development, so much rests on properly defining the desired outcome.
The problem is also that in the game of "helping people afford all their needs", governments will rarely construct solutions that involve less government. Which might well be the right answer.
Can you exemplify this?
Examples are impossible: governments are too complex. Any government that does this (however it is done), that government is also doing other things that are confounding factors.
The idea is simple: a government that does less costs less, so it needs to collect less taxes. That means people just over the edge could pay for their dental work instead of the government and all the overhead of having a third party involved.
It doesn't help those completely unable to care for themselves, but those who are close are no longer pushed over the edge.
The idea is simple: a government that does less costs less, so it needs to collect less taxes. That means people just over the edge could pay for their dental work instead of the government and all the overhead of having a third party involved.
It doesn't help those completely unable to care for themselves, but those who are close are no longer pushed over the edge.
Deregulation of airlines led to the modern era of cheap air travel.
That may have helped, but you can't ignore other factors such as relatively stable inflation adjusted fuel costs, combined with better fuel efficiency, lowering operating costs, or the trend towards eliminating amenities and increasing passenger density. Or the continued increase in airline safety while overseen by the not deregulated role of the FAA. Deregulation was only part of the picture.
Deregulation of energy companies led to Enron.
What exactly did you just prove by this overly generalising statement?
What exactly did you just prove by this overly generalising statement?
[deleted]
[deleted]
"less gameable policy" -> clinics do not get a blank check, for good or bad, they have to comply with government's bureaucracy requirements, i.e., explicitly take action that attests that the clinic considers the procedure to be necessary and covered under that particular government aid. That is, if the clinic does the bureaucratic 'work' when the procedure would not have been recommended otherwise, their actions border: fraud and healthcare crime.
"The clinics had the incentive to “fix teeth for people in need” - so they did", in the case at hand, being the procedures unnecessary, the incentive was not "fix teeth for people in need", but "increase profits".
"The clinics had the incentive to “fix teeth for people in need” - so they did", in the case at hand, being the procedures unnecessary, the incentive was not "fix teeth for people in need", but "increase profits".
I agree completely that “increase profits” was a goal for the clinics, but I mean something a little different by incentive. I mean “a behavior that is being encouraged or rewarded by others.”
You are asserting that the government’s policy was sufficiently specific regarding “medical necessity” that would clearly show these claims would not qualify. If that were the case, then the government itself should not have paid out those claims in the first place. The fact that they did pay the claims (at least until someone blew a whistle) suggests either that their policy was vague enough to not be so enforceable, or that their review and enforcement was lax (or both).
That’s my point - if the policy and enforcement had been clear from the get go this wouldn’t have been so big a deal. One or two bad apples try something and get sanctioned and the whole thing stops being an incentive to bad actors.
We agree that something changed in the situation such that this is no longer occurring, right? What was that change, and why was it not so in the first place?
You are asserting that the government’s policy was sufficiently specific regarding “medical necessity” that would clearly show these claims would not qualify. If that were the case, then the government itself should not have paid out those claims in the first place. The fact that they did pay the claims (at least until someone blew a whistle) suggests either that their policy was vague enough to not be so enforceable, or that their review and enforcement was lax (or both).
That’s my point - if the policy and enforcement had been clear from the get go this wouldn’t have been so big a deal. One or two bad apples try something and get sanctioned and the whole thing stops being an incentive to bad actors.
We agree that something changed in the situation such that this is no longer occurring, right? What was that change, and why was it not so in the first place?
In my experience, people who are paying the bill tend to ask more questions and be more skeptical the higher the price is.
The government is asking those questions, that is the "work" the clinics do when they say they "would do the work" to be able to bill the government for such procedures.
Yes, but this skepticism is problematic when it comes to healthcare. Particularly when it comes to high-deductible plans -- people are too skeptical, and will avoid seeking the care they need. Even when the cost of that early care is less than what they would otherwise be playing in premiums.
edit: retracted: (sometimes leading to more expensive measures needed down the road) — see replies
edit: retracted: (sometimes leading to more expensive measures needed down the road) — see replies
I wouldn't be so quick to retract: According to that wikipedia link, while costs may not have changed much, mental & physical health were improved. Essentially, the cost per-unit of overall patient wellbeing was decreased because costs remained the same while patient care increased.
The Oregon Medicaid expansion didn’t reach this conclusion. On balance people were happier because of lower stress, and people with diabetes were better served, but there wasn’t a huge macro effect of cost reduction via early intervention.
We already know the thing about diabetes, to the point that even private insurance companies will throw money at compliance because it’s cheaper than care. Outside that, vaccinations, and maybe a few more specific cases, the conventional wisdom of early intervention saves money is false.
https://en.m.wikipedia.org/wiki/Oregon_Medicaid_health_exper...
We already know the thing about diabetes, to the point that even private insurance companies will throw money at compliance because it’s cheaper than care. Outside that, vaccinations, and maybe a few more specific cases, the conventional wisdom of early intervention saves money is false.
https://en.m.wikipedia.org/wiki/Oregon_Medicaid_health_exper...
patients also had a "higher self-reported mental and physical health"
I think that's a heck of an important thing to mention here. Costs didn't change much, but health was improved.
I think that's a heck of an important thing to mention here. Costs didn't change much, but health was improved.
Thanks for the link, I edited my comment
The price of dental work is not very high. A few thousand dollars would cause you or me to ask questions, but the government deals with billions, so that little bit gets lost in the noise. It can make a lot of many for those in the scam without being easy to notice.
I think this is a good example of why in-kind benefits programs are a problem, and if you want to relieve the effects of poverty you should just give people money.
There was a recent UK scandal over the food boxes being handed out as a substitute for free school meals while schools are closed; it was immediately obvious to anybody who has ever shopped for their own groceries that the intermediary company was taking the £30, skimming off most of it, and providing a deeply inadequate result: https://www.dw.com/en/food-parcels-for-kids-spark-outrage-in...
(I note that even in the British NHS, dental and optical services are not completely free at the point of use: https://www.nhs.uk/nhs-services/dentists/dental-costs/unders... )
There was a recent UK scandal over the food boxes being handed out as a substitute for free school meals while schools are closed; it was immediately obvious to anybody who has ever shopped for their own groceries that the intermediary company was taking the £30, skimming off most of it, and providing a deeply inadequate result: https://www.dw.com/en/food-parcels-for-kids-spark-outrage-in...
(I note that even in the British NHS, dental and optical services are not completely free at the point of use: https://www.nhs.uk/nhs-services/dentists/dental-costs/unders... )
I think you just made a case for why news outlets shouldn't write articles based on random tweets without verifying anything- that lie is now fact in some peoples minds:
> Chartwells, the school catering company that supplied the package, claimed the photograph showed five days of free lunches, not 10, and the charge for food, packing and distribution was £10.50 not £30,
https://www.theguardian.com/education/2021/jan/12/food-parce...
> Chartwells, the school catering company that supplied the package, claimed the photograph showed five days of free lunches, not 10, and the charge for food, packing and distribution was £10.50 not £30,
https://www.theguardian.com/education/2021/jan/12/food-parce...
That's what they claim, but is that actually true?
Anyway, the campaign got the policy changed; Starmer went and looked at the DFE guidence to check that the expected parcel really was that bad. https://hansard.parliament.uk/commons/2021-01-13/debates/C80...
Anyway, the campaign got the policy changed; Starmer went and looked at the DFE guidence to check that the expected parcel really was that bad. https://hansard.parliament.uk/commons/2021-01-13/debates/C80...
The problem is the poor I want to help most are the poor who cannot take care of themselves. Those who can take care of the money (as opposed to giving it to scammers) can also figure out how to get more and just need the government out of the way. (this is the conservative lower taxes point). What is left is a lot of people with mental issues who can never take care of themselves and wouldn't use the money to do it. The money needs to go to some ??? that will take care of those people.
It's hard to understand how you can take this point of view if you've ever met an average poor person. (The "poor who cannot take care of themselves" are a minority who require different support)
One definition of poor is that people do not have access to enough money to meet their needs.
And, poor people cannot meet the needs of getting more money: tasks such as education, job search, etc, are limited by an immediate need to get rent, food, and warmth.
If the government "gets out of the way", the safety net of being near-guaranteed to cover rent, food and warmth, will also quickly get out of the way, exacerbating poor peoples' problems.
One definition of poor is that people do not have access to enough money to meet their needs.
And, poor people cannot meet the needs of getting more money: tasks such as education, job search, etc, are limited by an immediate need to get rent, food, and warmth.
If the government "gets out of the way", the safety net of being near-guaranteed to cover rent, food and warmth, will also quickly get out of the way, exacerbating poor peoples' problems.
As Adam Smith observed, excess money tends to go into rent. The poor in India sleep on a sidewalk. The poor in the US have a 2 bedroom apartment with running water and a TV. Both sets struggle to make ends meet (I struggle to make ends meet sometimes, and I'm clearly well above average in the upper middle class with 6 months of emergency savings that I dare not touch), so struggle to make ends meet is NOT what it means to be poor.
Everybody will try to make their lives better. Anyone who isn't careful will end up spending so much on little things that they can't afford the basics at the end of the month. I've been poor (for the US - a much higher standard than poor elsewhere!), and I have no desire to go back.
I will agree the poor who cannot take care of themselves are a minority. However I maintain that given money to the rest won't help because they will spend it on things like a nicer apartment. Worse I fear that their attempt to get a nicer apartment will make rent go up and in the end they spend more money on the same apartment - thus making that extra money an indirect handout to the rich.
Everybody will try to make their lives better. Anyone who isn't careful will end up spending so much on little things that they can't afford the basics at the end of the month. I've been poor (for the US - a much higher standard than poor elsewhere!), and I have no desire to go back.
I will agree the poor who cannot take care of themselves are a minority. However I maintain that given money to the rest won't help because they will spend it on things like a nicer apartment. Worse I fear that their attempt to get a nicer apartment will make rent go up and in the end they spend more money on the same apartment - thus making that extra money an indirect handout to the rich.
> The poor in India sleep on a sidewalk. The poor in the US have a 2 bedroom apartment...
And yet... the USA ranks higher than India in homeless people per capita, and also for street children.
> However I maintain that given money to the rest won't help because they will spend it on things like a nicer apartment.
And an appropriate level of heating, and healthier food, maybe. What's wrong with that?
And yet... the USA ranks higher than India in homeless people per capita, and also for street children.
> However I maintain that given money to the rest won't help because they will spend it on things like a nicer apartment.
And an appropriate level of heating, and healthier food, maybe. What's wrong with that?
> excess money tends to go into rent
Rent-seekers will attempt to extract the maximum they can get away with. The solution is controls to prevent that. Anything that is essential (rent, utilities, food) should not be subject to the whims and greed of private interests.
Rent-seekers will attempt to extract the maximum they can get away with. The solution is controls to prevent that. Anything that is essential (rent, utilities, food) should not be subject to the whims and greed of private interests.
Are you proposing to repeal the law of supply and demand?
It's unimaginative to say "we cannot do anything about this so let's do nothing".
Plenty of countries' and cities' policies are successfully curbing rents and landlordism - why are you ignoring empirical evidence to make your point?
Plenty of countries' and cities' policies are successfully curbing rents and landlordism - why are you ignoring empirical evidence to make your point?
> Those who can take care of the money can also figure out how to get more and just need the government out of the way. (this is the conservative lower taxes point)
Yes, and it's evidence-free nonsense, because many of those people are either not earning at all, often for very good reasons, or earning below the lower tax threshold. You might be able to make a case for abolishing sales tax/VAT, but at least in the UK food is already exempt from VAT.
Yes, and it's evidence-free nonsense, because many of those people are either not earning at all, often for very good reasons, or earning below the lower tax threshold. You might be able to make a case for abolishing sales tax/VAT, but at least in the UK food is already exempt from VAT.
That’s a dismissive attitude. There’s plenty of people who are poor through flaws in the system and no fault of their own, and there’s also plenty who are stuck in patterns of addiction where free money will exacerbate that.
I can’t see my way to blaming government for this. Either these operations were indicated but unlikely to happen otherwise (the government program was effective) or not medically-indicated (at which point private clinics are torturing poor people for profit which is abhorrent on so many levels).
You can't write a bureaucratic regulation that accurately replaces a judgement call.
For example, I have various dental problems. With each one, the dentist would lay out the options - effectiveness, how long they'll last, the price, how painful the procedure is, what the recovery time is, etc. There is no right answer. Different people make different choices.
(I've often picked an option that wasn't the dentist's recommended one.)
Medicine is almost never (Condition A => Treatment B).
For example, I have various dental problems. With each one, the dentist would lay out the options - effectiveness, how long they'll last, the price, how painful the procedure is, what the recovery time is, etc. There is no right answer. Different people make different choices.
(I've often picked an option that wasn't the dentist's recommended one.)
Medicine is almost never (Condition A => Treatment B).
It doesn't seem like the regulation was ever intended to replace that judgement call; just support the doctors' decision. We are not machines ruled by logic, and in the end everything that ends up in the justice system is judged by intention and/or merit and not 'the letter of the law'.
Is that true though? If something is technically legal like this, they get found innocent. When its technically illegal, mandatory minimums insure they go to jail for far too long. Someone once said there are legal traditions where intent of the law could be used to find guilt, and it had a name, but the US and UK are much more like machines.
I believe both the US and UK (as do most western countries?) have 'common law' systems, where jurisprudence / case precedents are heavily relied upon. Whenever there is doubt on the legality of something, that court decision will set the tone for others to follow, and go higher up the justice system as needed.
The main point was that you can't simply blame the law for failing to 'replace judgement'. Most laws rely on some amount of common sense to be interpreted.
The main point was that you can't simply blame the law for failing to 'replace judgement'. Most laws rely on some amount of common sense to be interpreted.
> I can’t see my way to blaming government for this.
They get to take money from people backed by the threat of locking them in a box if they don't. They are responsible for making sure that money is spent extremely effectively.
They get to take money from people backed by the threat of locking them in a box if they don't. They are responsible for making sure that money is spent extremely effectively.
The point isn't that the clinics are not morally culpable, but only that they are a natural response to government incentives over which the government has exclusive control.
Doing something morally wrong is not a natural response. It is a natural response option. To paraphrase Jurassic Park: [Clinics] were so preoccupied with whether or not they could, they didn’t stop to think if they should.
Not everyone chooses to do something morally wrong, just because "I'm allowed to! And it makes life easier!"
Not everyone chooses to do something morally wrong, just because "I'm allowed to! And it makes life easier!"
> Not everyone chooses to do something morally wrong, just because "I'm allowed to! And it makes life easier!"
But plenty choose to do something morally wrong because "I have to put bread on my table, and I'm about to go out of business because I can't compete with everyone else doing the immoral thing". This is the mechanism by which the market eliminates ethics from our lives, step by step. If doing something slightly wrong gives you a competitive advantage, you'll take it or you'll get outcompeted by everyone else who do.
But plenty choose to do something morally wrong because "I have to put bread on my table, and I'm about to go out of business because I can't compete with everyone else doing the immoral thing". This is the mechanism by which the market eliminates ethics from our lives, step by step. If doing something slightly wrong gives you a competitive advantage, you'll take it or you'll get outcompeted by everyone else who do.
How does one define "morally wrong"?
Honest business by eastern Massachusetts standards would have you labeled a swindler in southern Indiana.
There are large swaths of the US where "if the option is there then there's no shame in taking it" and "if you don't cash in you're only hurting yourself because someone else will" are just how normal business is done.
Honest business by eastern Massachusetts standards would have you labeled a swindler in southern Indiana.
There are large swaths of the US where "if the option is there then there's no shame in taking it" and "if you don't cash in you're only hurting yourself because someone else will" are just how normal business is done.
The government incentives were clearly going to result in private-island driven dinosaur experiments. If the incentives had been to welcome such genetic experimentation in a controlled manner then regulators would have noticed the predatory behaviour of the velociraptors before it became a problem.
Again, it's about risk/incentive management, not about judging those who dare to go too far. The T-Rex will eat you even as you shake your fist at its unethical re-creator. Better to not merely hope that everyone chooses the friendliest route by virtue of human nature.
Again, it's about risk/incentive management, not about judging those who dare to go too far. The T-Rex will eat you even as you shake your fist at its unethical re-creator. Better to not merely hope that everyone chooses the friendliest route by virtue of human nature.
You are absolutely right - it’s an option.
If we keep setting up situations where people have an unethical option, I’m not sure we should act surprised by them taking it. We can, it’s an option, but it seems naive.
The government had the option to think through their policymaking more fully. What accountability should they have for that?
If we award points for “trying, with good intentions”, and not for “actual outcome” we are in for a lot more bad outcomes from earnest triers.
If we keep setting up situations where people have an unethical option, I’m not sure we should act surprised by them taking it. We can, it’s an option, but it seems naive.
The government had the option to think through their policymaking more fully. What accountability should they have for that?
If we award points for “trying, with good intentions”, and not for “actual outcome” we are in for a lot more bad outcomes from earnest triers.
Who defines “medically indicated” - the government, or the clinicians?
The modern business maxim "don't leave money on the table" leads to local maxima, the locus being your own self in the short term. If the clinics self-regulate, don't artificially inflate the number of procedures needed and ostracize clinics who do, then everyone wins in the long run. The alternative is, everyone attempts to maximize gain and the government has to eventually step in with more draconian regulations that hurt everyone. Rational, local altruism can be a path to global self-interest.
> The modern business maxim
Not only has it always been a business maxim, it's a maxim for pretty much every organization and 80% of the population. (I'm being generous, I suspect it is more like 95%.)
Trying to construct a society that relies on people being selfless most of the time is just never going to work.
Not only has it always been a business maxim, it's a maxim for pretty much every organization and 80% of the population. (I'm being generous, I suspect it is more like 95%.)
Trying to construct a society that relies on people being selfless most of the time is just never going to work.
> But blaming the clinics doesn't work, because the actual problem was the government set things up so that the rise of clinics to exploit the rules was legal and inevitable.
The obvious solution would be for the government to punish the clinics for acting against the spirit of the law. But that approach brings in a huge potential for abuse. We're in a bind.
If we can solve this, this will be the biggest breakthrough for civilization ever since the invention of hierarchical structures many millennia ago. Hierarchies and codified laws were what allowed us to stop living as isolated groups of ~150 people[0] and go on to build a global civilization. But these came at the cost of not being able to exploit our innate social skills to cooperate with each other. Bureaucracies, rules and metrics are thus an inefficient and barely adquate mechanical substitute for trust.
If we could somehow scale trust, scale social skills, so that people would naturally respond to incentives that are uncomputable and not expressible through bureaucratic apparatus - like "don't do stupid shit", or "really, don't cheat" - we'd enter a golden age.
Alas, I have no clue how to do that.
--
[0] - Yes, the Dunbar's number. Past which you no longer have the case that everyone knows everyone else, and interpersonal dynamics stop being strong enough to ensure mutual cooperation.
The obvious solution would be for the government to punish the clinics for acting against the spirit of the law. But that approach brings in a huge potential for abuse. We're in a bind.
If we can solve this, this will be the biggest breakthrough for civilization ever since the invention of hierarchical structures many millennia ago. Hierarchies and codified laws were what allowed us to stop living as isolated groups of ~150 people[0] and go on to build a global civilization. But these came at the cost of not being able to exploit our innate social skills to cooperate with each other. Bureaucracies, rules and metrics are thus an inefficient and barely adquate mechanical substitute for trust.
If we could somehow scale trust, scale social skills, so that people would naturally respond to incentives that are uncomputable and not expressible through bureaucratic apparatus - like "don't do stupid shit", or "really, don't cheat" - we'd enter a golden age.
Alas, I have no clue how to do that.
--
[0] - Yes, the Dunbar's number. Past which you no longer have the case that everyone knows everyone else, and interpersonal dynamics stop being strong enough to ensure mutual cooperation.
I agree with what you write, but I'm also spitballing stopgap measures: Are there downsides to having governments instead contract healthcare capacity? Has this been tried?
By this I mean that instead of the government paying each hospital (or in this case dentist) for each procedure, the government pays each hospital (or dentist) to keep a certain capacity for each kind of procedure available. The hospitals are obliged to provide up to that capacity at no extra cost to the government. Any demand beyond it, goes into the backlog (and turns into a political issue).
Surely I'm missing some downsides…
By this I mean that instead of the government paying each hospital (or in this case dentist) for each procedure, the government pays each hospital (or dentist) to keep a certain capacity for each kind of procedure available. The hospitals are obliged to provide up to that capacity at no extra cost to the government. Any demand beyond it, goes into the backlog (and turns into a political issue).
Surely I'm missing some downsides…
We do this where I work (municipal social work and housing with care) and it really does remove some of the incentives to up-code and over-produce on the part of the provider in the short term.
In the long term there's still some incentive to take on extra clients for the care providers so they have a better negotiating position for their next contact.
In the long term there's still some incentive to take on extra clients for the care providers so they have a better negotiating position for their next contact.
I think you might run into the issue of whether or not that capacity is actually being provided or just reported. You can definitely track that a procedure was done: you have the patient that you can contact to verify, bills for the materials that were needed, and bookings that guarantee no one was operating on multiple patients at once. If you just pay for capacity regardless of whether or not it's used, how do you know that they aren't just turning half of all patients away at the door and claiming that they provide the full capacity? There's political unrest about not enough capacity but your healthcare providers are saying no one's coming in and still taking the full amount of money.
> I have no clue how to do that.
Free markets are remarkable in their ability to get people to cooperate in their mutual self-interest.
Free markets are remarkable in their ability to get people to cooperate in their mutual self-interest.
The free market is a really simple feedback system that, in any real-world implementation, can just as quickly lose the "mutual self-interest" part, or replace "cooperation" with "coercion". The very problem with the clinics you described - that's the free market in action.
The problem here is that there is no way for society at large to send corrective feedback, "holy shit you're totally abusing the rules and what you do is immoral", and have it actually impact said clinics' behavior. This mechanism exists naturally in small groups, but money and scale destroys it.
The problem here is that there is no way for society at large to send corrective feedback, "holy shit you're totally abusing the rules and what you do is immoral", and have it actually impact said clinics' behavior. This mechanism exists naturally in small groups, but money and scale destroys it.
> that's the free market in action
Government paying for things is not the free market in action.
Government paying for things is not the free market in action.
In your story, government acts just as a customer on the market. Mere presence of a government doesn't make a market not free.
Government is not spending its own money but that of taxpayers present and future - an agency-style conflict of interest. This is then made worse by the fact that the party who receives the services purchased (the patient) is someone different again who has no skin in the game at all.
This is a recipe for market failure and not something I'd describe as a "free" market, even if it is a market.
This is a recipe for market failure and not something I'd describe as a "free" market, even if it is a market.
> Government is not spending its own money but that of taxpayers present and future - an agency-style conflict of interest.
The government is spending its own money it collected in taxes[0]. Whether it is more like a thug or more like an embodiment of the collective will of society is immaterial. It's just another customer on the market.
> This is then made worse by the fact that the party who receives the services purchased (the patient) is someone different again who has no skin in the game at all.
This is the exact same scenario as an advertiser paying a company to expose me to advertising; I have no skin in that game either, and I'm subjected to something I do not necessarily want. But nobody would say it's not "free market". Hell, in the case of the clinics, the situation would be identical if the party paying for the procedures was not the government, but a billionaire philanthropist.
In fact, nothing about a free market precludes such scenarios, where the choices available to a buyer are dependent on the deals made between the seller and some third parties.
--
[0] - The concept that "government has no money" is silly outside the quite obvious reminder that government is mostly funded by its subjects.
The government is spending its own money it collected in taxes[0]. Whether it is more like a thug or more like an embodiment of the collective will of society is immaterial. It's just another customer on the market.
> This is then made worse by the fact that the party who receives the services purchased (the patient) is someone different again who has no skin in the game at all.
This is the exact same scenario as an advertiser paying a company to expose me to advertising; I have no skin in that game either, and I'm subjected to something I do not necessarily want. But nobody would say it's not "free market". Hell, in the case of the clinics, the situation would be identical if the party paying for the procedures was not the government, but a billionaire philanthropist.
In fact, nothing about a free market precludes such scenarios, where the choices available to a buyer are dependent on the deals made between the seller and some third parties.
--
[0] - The concept that "government has no money" is silly outside the quite obvious reminder that government is mostly funded by its subjects.
I'm not trying to make a moral point about whether taxes are right or wrong, simply that people behave differently when spending other people's money rather than their own. Government spending taxpayers' money is exactly the same conflict in this sense as company management spending shareholders' money. These agency conflicts can an do often lead to bad outcomes and when this happens it's generally viewed as a form of market failure.
Considering you are talking about the health care system where insurance companies spend money on behalf of their customers. At no point did the customer of the doctor or dentist spend his or her hard earned money. It was always a large, authoriative corporation.
Why would it matter if the corporation is the government or an insurance company? Also, are you implying that you can't have a free market if you have insurance companies?
Why would it matter if the corporation is the government or an insurance company? Also, are you implying that you can't have a free market if you have insurance companies?
It is indeed a problem with insurance as well where it's more commonly called moral hazard. With insurance traditionally there are a number of mechanisms which can help mitigate this eg copays, deductibles, etc. - and there is (sometimes) a market for pricing of policy premiums and terms.
But it's far from perfect, especially in the US (at least as I understand the American system).
But it's far from perfect, especially in the US (at least as I understand the American system).
I'm also not trying to make a moral point. I'm trying to show that these kinds of problems are one of the possible failure modes on the free market, and whether one of the actors happens to be a government is immaterial - you could (like you personally just did) draw an equivalent scenario involving only private businesses.
When people talk about "free" markets it usually means markets where these types of problems are effectively managed or mitigated, not "markets with no rules".
There's of course lots of debate about what the right policies to manage and mitigate these are but it doesn't mean that markets aren't a good mechanism just because there might be trade-offs or imperfect solutions.
There's of course lots of debate about what the right policies to manage and mitigate these are but it doesn't mean that markets aren't a good mechanism just because there might be trade-offs or imperfect solutions.
I always understood "free" markets to be ones run by supply and demand, without authoritarian interference. That does not exclude principal-agent problems, though.
I'm also not saying markets aren't a good idea. Just I don't believe they're the core of the solution to the problem I outlined upthread.
I'm also not saying markets aren't a good idea. Just I don't believe they're the core of the solution to the problem I outlined upthread.
The government’s money are unearned money, confiscated from taxpayers. There is a saying about this kind of income: "easy come, easy go". You can be sure that "the subjects" this money was taken from would be much more careful about spending it.
Not convinced? Just look at the way government borrows: no sane person would sell her future like that. Also, just ask a politician how worried he is about spending money "for his constituents". I know stadiums made for tiny towns, boulevards for undriven neighborhoods, and so on and so forth...
Not convinced? Just look at the way government borrows: no sane person would sell her future like that. Also, just ask a politician how worried he is about spending money "for his constituents". I know stadiums made for tiny towns, boulevards for undriven neighborhoods, and so on and so forth...
Sure. But that's neither here nor there, as the market is an equal opportunity institution. It doesn't ask whether you got your money by providing goods in exchange, or confiscated it from your subjects, or won it in a casino, or robbed a bank, or stole it from your employees' pension fund, ...
It is there, since such a large player as a government throwing around unlimited money will inevitably distort and corrupt the market.
Look at what is going on in the stock market right now since the fed started buying assets in spring.
Look at what is going on in the stock market right now since the fed started buying assets in spring.
Sure, but that does not apply to the case we're discussing - which is a government paying clinics for procedures. You could substitute "a billionaire philanthropist" for "a government" and the outcome would be exactly the same.
Now if you want to talk free markets in general, then I want to point out that in the real world, a free market cannot thrive without some sort of government that ensures social stability and enforces contracts. Without such government, the market would quickly degenerate into trading violence alongside currency, and either disintegrate or... end up producing a government. Ultimately, "private enterprise" and "public governance" don't have a clear boundary between them - they're two modes of the same phenomenon, people negotiating with each other.
Now if you want to talk free markets in general, then I want to point out that in the real world, a free market cannot thrive without some sort of government that ensures social stability and enforces contracts. Without such government, the market would quickly degenerate into trading violence alongside currency, and either disintegrate or... end up producing a government. Ultimately, "private enterprise" and "public governance" don't have a clear boundary between them - they're two modes of the same phenomenon, people negotiating with each other.
Free markets can only exist in this way until they are much smaller than the whole economy. The moment they are the economy they are heavily prone to monopolisation.
On the contrary, the larger the market, the more the competition and thus the less danger of monopolization.
Wealth begets wealth. It compounds. The larger the market, the more space there is for a company to take the lead, put up barriers to entry, and focus on snuffing out any would-be competitors.
Only governments can put barriers of entry into a free market, usually through regulations.
Not true. See [0] for a list of examples, which contains several cases of barriers to entry that do not involve governments in any way (except as the entity that allows contracts to be enforced and the market to exist in the first place).
--
[0] - https://en.wikipedia.org/wiki/Barriers_to_entry#Examples
--
[0] - https://en.wikipedia.org/wiki/Barriers_to_entry#Examples
Half of those examples need to be sustained by the government to actually act as barriers. And all of them ignore the creativity of the entrepreneurial mind when incentivized by the fabulous profits to be had in a monopolized market. A nice example:
https://en.wikipedia.org/wiki/Herbert_Henry_Dow
https://en.wikipedia.org/wiki/Herbert_Henry_Dow
I sometimes feel the same about my dental coverage; my insurance will compensate something like €500 a year and the dentist knows this (I think it shows up on the insurance records they have access to). Mind you, my teeth and dental hygiene isn't the best so I'd rather they do the work than lose my teeth.
> the actual problem was the government set things up so that the rise of clinics to exploit the rules was legal and inevitable.
https://en.wikipedia.org/wiki/Cobra_effect
https://en.wikipedia.org/wiki/Cobra_effect
Medical insurance fraud occurs with private insurance as well.
Blaming the government here is a bit like blaming a life insurance company when someone murders a relative for the insurance money.
Blaming the government here is a bit like blaming a life insurance company when someone murders a relative for the insurance money.
"When a measure becomes a target, it ceases to be a good measure"
commonly known as Goodhart's law (https://en.wikipedia.org/wiki/Goodhart%27s_law) although credits should go to Marilyn Strathern for this formulation.
commonly known as Goodhart's law (https://en.wikipedia.org/wiki/Goodhart%27s_law) although credits should go to Marilyn Strathern for this formulation.
Seems like one of the side effects here is that a lot of people got dental work that wouldn't have otherwise done so. Isn't that a positive outcome worth something? If it's worth about as much (to the patients, and to society) as what was paid, maybe it wasn't as wasteful as at first blush?
> the actual problem was the government set things up
edit: so apparently this anecdote was in the article... I guess I just gave away that I commented before I read the article...
Reminds me of the cobra problem in India. After England colonized India, they noticed that India had a cobra problem, so they created a bounty for cobras: bring a dead cobra, get paid. So people, of course, started farming cobras to get paid by the English government. Once the English government figured out what was going on, they scrapped the bounty program and the cobra farmers released all of their now-worthless cobras into the wild... increasing the cobra problem.
edit: so apparently this anecdote was in the article... I guess I just gave away that I commented before I read the article...
Reminds me of the cobra problem in India. After England colonized India, they noticed that India had a cobra problem, so they created a bounty for cobras: bring a dead cobra, get paid. So people, of course, started farming cobras to get paid by the English government. Once the English government figured out what was going on, they scrapped the bounty program and the cobra farmers released all of their now-worthless cobras into the wild... increasing the cobra problem.
If you're measuring an activity, that's where you end up. People will do more of that activity regardless of outcome. The clinics should be rewarded for the actual outcome you want: better dental health in a certain population.
> But blaming the clinics doesn't work, because the actual problem was the government set things up so that the rise of clinics to exploit the rules was legal and inevitable.
Actually, blaming the clinics does does work, or at least is appropriate. They're the ones that are doing unnecessary work. Just because you can do something does not mean you should do it.
I person can cheat on their spouse, but should they? Was it the spouse's fault that they let the person go on a business trip/conference unaccompanied? The cheated-on spouse should have know better and set up a system to prevent it from happening.
Actually, blaming the clinics does does work, or at least is appropriate. They're the ones that are doing unnecessary work. Just because you can do something does not mean you should do it.
I person can cheat on their spouse, but should they? Was it the spouse's fault that they let the person go on a business trip/conference unaccompanied? The cheated-on spouse should have know better and set up a system to prevent it from happening.
The dental clinics do this to everyone, not just the poor.
It's an issue with all forms of health care. If a hospital makes money for each test done, the financial incentive will be to run a lot of tests.
It's an issue with all forms of health care. If a hospital makes money for each test done, the financial incentive will be to run a lot of tests.
> The dental clinics do this to everyone
I had a dentist pull that on me, I told him off, and went to another one. (He wanted to pull my wisdom teeth, I told him there was nothing wrong with them, and there still isn't 40 years later. Not even a cavity.)
I had a dentist pull that on me, I told him off, and went to another one. (He wanted to pull my wisdom teeth, I told him there was nothing wrong with them, and there still isn't 40 years later. Not even a cavity.)
> It's an issue with all forms of health care. If a hospital makes money for each test done
That doesn't line up. In some countries, such as the UK, doctors are generally salaried employees.
With that said, I don't think the same is true of UK dentists.
That doesn't line up. In some countries, such as the UK, doctors are generally salaried employees.
With that said, I don't think the same is true of UK dentists.
In some countries, such as the UK, doctors are generally salaried employees.
This is not true. A GP practice is a private partnership like a legal chambers which bills the NHS for its services. This fact is strangely overlooked by people angry about “privatising the NHS”, it’s always been this way.
This is not true. A GP practice is a private partnership like a legal chambers which bills the NHS for its services. This fact is strangely overlooked by people angry about “privatising the NHS”, it’s always been this way.
Some UK GPs are salaried. I don't know the proportions.
https://www.healthcareers.nhs.uk/explore-roles/doctors/pay-d...
https://www.healthcareers.nhs.uk/explore-roles/doctors/pay-d...
Some UK GPs are salaried. I don't know the proportions.
Paid a salary by the private practice they work for, again just like a junior lawyer would be.
Paid a salary by the private practice they work for, again just like a junior lawyer would be.
"the government did this". This is a perfect example of how brainwashed Americans are about 'government is the problem' they will gloss over unethical medical malpractice by professionals and blame 'the government' instead of being a little introspective about why a shitty unethical culture like this is not only allowed to continue but in fact expected. Its as if citizenship is some archaic ideal from a long-gone era to be replaced with rational utilitarianism where every actor is cynically out to screw over everyone else. and in the next sentence, as if aiming to create the perfect non sequitur, bemoan why 'management by metrics' does not work
There are lots of perverse incentives in dentistry.
I've had some dentists do the "mouth mining" kind of thing.
I have my four wisdom teeth, and one dentists was immediately "We should schedule you to have those wisdom teeth out." He's not the only one that asked, but "why should they come out?" always has these vague answers, like "might cause crowding".
So, I've had my wisdom teeth all my adult and they are sort of a litmus test for dentists.
I've heard navy seals require them out, I thought for some deep water diving reason.
I've had some dentists do the "mouth mining" kind of thing.
I have my four wisdom teeth, and one dentists was immediately "We should schedule you to have those wisdom teeth out." He's not the only one that asked, but "why should they come out?" always has these vague answers, like "might cause crowding".
So, I've had my wisdom teeth all my adult and they are sort of a litmus test for dentists.
I've heard navy seals require them out, I thought for some deep water diving reason.
[deleted]
A bit of a segue, but this management by metrics goes beyond just revenue.
KPI, KRI, KCI, etc. Truly a heap of metrics that eat out the time of middle management, and consequently of the people down the hierarchical pyramid. It over-allocates resources towards resource management itself rather than towards products themselves.
There is a drive to justify your existence, and that of your fellow workers, when your company becomes large enough that it has bureaucratized and, moreover, acquired some form of legacy.
David Graeber in Bullshit Jobs mentions that two categories of BS were the box-tickers and the taskmasters. They are evidence of this management by metrics.
It's all about budget: gotta spend that hard-earned budget to find ways to save budget on your budget-consuming core products... to make sure your budget allocation is the same next month or year.
KPI, KRI, KCI, etc. Truly a heap of metrics that eat out the time of middle management, and consequently of the people down the hierarchical pyramid. It over-allocates resources towards resource management itself rather than towards products themselves.
There is a drive to justify your existence, and that of your fellow workers, when your company becomes large enough that it has bureaucratized and, moreover, acquired some form of legacy.
David Graeber in Bullshit Jobs mentions that two categories of BS were the box-tickers and the taskmasters. They are evidence of this management by metrics.
It's all about budget: gotta spend that hard-earned budget to find ways to save budget on your budget-consuming core products... to make sure your budget allocation is the same next month or year.
so you want management without metrics? the problem isn't the metrics (a fundamental attribution error to think so), it's that middle managers are incentivized to appease upper managers (and ultimately owners), and metrics are the most visible available tool. the system is set up this way intentionally by owners and their executive surrogates to insulate them from the callous, negative conqeuences of management that explicitly aims to dehumanize work (this kind of indirection abstraction in human systems is an extremely common source of distorted incentives and unintended consequences; like poor customer service at most companies, google being a preeminent example).
metrics that are reported to individual contributors (e.g., toyota production system, agile done appropriately), on the other hand, are fantastic, for managers and workers, as long as it's well-understood that the metrics are not the product, they're simple signposts (indirect/incomplete measures of holistic qualitative desires like efficiency and quality).
by the way, (bad) bureaucracies are created by the drive to regiment and order messy human processes that leads to over-reliance on metrics, not people trying to justify their existence.
bullshit jobs is a bullshit book written to appeal to our fears and frustrations, not to elucidate us with actionable information.
metrics that are reported to individual contributors (e.g., toyota production system, agile done appropriately), on the other hand, are fantastic, for managers and workers, as long as it's well-understood that the metrics are not the product, they're simple signposts (indirect/incomplete measures of holistic qualitative desires like efficiency and quality).
by the way, (bad) bureaucracies are created by the drive to regiment and order messy human processes that leads to over-reliance on metrics, not people trying to justify their existence.
bullshit jobs is a bullshit book written to appeal to our fears and frustrations, not to elucidate us with actionable information.
Absence of metrics would be shortsighted. Metrics with a purpose other than surveillance and budget would be good.
However, most metrics I have encountered and helped impleme t in the past, especially in the finance industry, fall in that category. Boxticking and task-management because there is a mentality of cutting corners.
Bullshit Jobs is an essay, not a research paper. It points at a globally felt phenomenon in the workplace. People's impressions, no matter how subjective, still imply something is afoot.
Also, a book pointing at an issue is not obligated in providing a solution. Criticizing a system has its own merit. It outlines an issue maybe someone else will then be able to elucidate.
However, most metrics I have encountered and helped impleme t in the past, especially in the finance industry, fall in that category. Boxticking and task-management because there is a mentality of cutting corners.
Bullshit Jobs is an essay, not a research paper. It points at a globally felt phenomenon in the workplace. People's impressions, no matter how subjective, still imply something is afoot.
Also, a book pointing at an issue is not obligated in providing a solution. Criticizing a system has its own merit. It outlines an issue maybe someone else will then be able to elucidate.
Truly a heap of metrics that eat out the time of middle management, and consequently of the people down the hierarchical pyramid. It over-allocates resources towards resource management itself rather than towards products themselves
You mean, it provides lots of avenues for the management class to provide cushy jobs for each other, at the expense of the real workers. Most organisations could ditch most of their middle management without skipping a beat.
You mean, it provides lots of avenues for the management class to provide cushy jobs for each other, at the expense of the real workers. Most organisations could ditch most of their middle management without skipping a beat.
We should do better than to be ignorant of national and global decisionmaking and responsibility in public health and pandemic epidemology.
It is a baseless claim that public health is systematically ignored in favor of pushing the pandemic’s reproduction number down. Do we really expect public health officers to be untrained, stupid, and dismissed? Is that what we really are seeing? Do we really think that public officials are ignorant of the metabolic and social implications of having to drive stakes into society?
It is a fact that in functioning states, public health decisions are taken within an ethical framework of how to do least harm and preserve the most freedom, considering resource availability and longer-term effects of public health decisions, including the interacting feedback loops of the economy and public health.
While overreliance on small metrics is clearly bad, this style of off-hand ignorance of public service is not what we need.
It is a baseless claim that public health is systematically ignored in favor of pushing the pandemic’s reproduction number down. Do we really expect public health officers to be untrained, stupid, and dismissed? Is that what we really are seeing? Do we really think that public officials are ignorant of the metabolic and social implications of having to drive stakes into society?
It is a fact that in functioning states, public health decisions are taken within an ethical framework of how to do least harm and preserve the most freedom, considering resource availability and longer-term effects of public health decisions, including the interacting feedback loops of the economy and public health.
While overreliance on small metrics is clearly bad, this style of off-hand ignorance of public service is not what we need.
>It is a baseless claim that public health is systematically ignored in favor of pushing the pandemic’s reproduction number down.
It's not baseless when there have been multiple studies now showing lockdowns don't help, and even the WHO now recommends against them, yet authorities still push for them.
It's not baseless when there have been multiple studies now showing lockdowns don't help, and even the WHO now recommends against them, yet authorities still push for them.
Can you point me at the WHO advice and how they're defining "lockdown"? And what the correct answer is? Because England is currently running at over 1600 COVID deaths a day, or half a 9/11, and we desperately need something that does help.
Where are you getting this number? NHS published data is less than half this number.
https://www.msn.com/en-gb/news/uknews/uk-reports-record-covi...
https://www.theguardian.com/world/2021/jan/20/uk-reports-182...
https://www.bbc.co.uk/news/uk-55738174
https://coronavirus.data.gov.uk/
Where are you getting your data from?
https://www.theguardian.com/world/2021/jan/20/uk-reports-182...
https://www.bbc.co.uk/news/uk-55738174
https://coronavirus.data.gov.uk/
Where are you getting your data from?
Straight from the source: https://www.england.nhs.uk/statistics/statistical-work-areas...
> deaths of patients who have died in hospitals in England
Not including deaths of patients who died outwith hospitals.
Not including deaths of patients who died outwith hospitals.
It does seem like a poorly chosen example. In the case of an exponential process, the doubling time is a remarkably important number.
Great article! I also found that the aim to be objective in everything and use metrics as truth and not just another input is pretty harmful over the long run in business and science. Things that are hard to measure are ignored most of the time.
The best example are apples. When you breed apples to be 3% larger each year but taste slightly worse, you can weigh them and see that they got slightly larger. As you cannot accurately measure taste you will notice that they basically taste the same. After years of breeding you have giant apples with bad taste.
The author's proposition of using as many metrics as possible and talking to real people is a good solution to make better decisions.
I would add that for some decisions you should actually just focus on the few facts that truly matter. The classic example being a potential business partner with a bad reputation. If somebody is not trustworthy everything else probably doesn't matter.
The best example are apples. When you breed apples to be 3% larger each year but taste slightly worse, you can weigh them and see that they got slightly larger. As you cannot accurately measure taste you will notice that they basically taste the same. After years of breeding you have giant apples with bad taste.
The author's proposition of using as many metrics as possible and talking to real people is a good solution to make better decisions.
I would add that for some decisions you should actually just focus on the few facts that truly matter. The classic example being a potential business partner with a bad reputation. If somebody is not trustworthy everything else probably doesn't matter.
I agree. Anything which only boils down to metrics would have already been automated.
Marketer here and this is so spot on. So many companies that I have worked for have had founders that just didn't seem to understand metrics as well as they thought they did.
Huge spread sheets with % and pivot tables and week on week breakdowns BUT no one could tell me much about what all that data meant. Apart from the fact that everyone had their own explanation it was just useless information to be fair. Churn, CAC, amongst various other 'hot' metrics.
In the end, most of these companies took a couple of years to realize that these metrics which Unicorns write amazing blog posts on are either BS or just not made for them. So much frustration was felt because a manager read a TechCrunch article about how X (with 100's of millions) did this and achieved something spectacular.
Huge spread sheets with % and pivot tables and week on week breakdowns BUT no one could tell me much about what all that data meant. Apart from the fact that everyone had their own explanation it was just useless information to be fair. Churn, CAC, amongst various other 'hot' metrics.
In the end, most of these companies took a couple of years to realize that these metrics which Unicorns write amazing blog posts on are either BS or just not made for them. So much frustration was felt because a manager read a TechCrunch article about how X (with 100's of millions) did this and achieved something spectacular.
I don’t know if there’s a phrase for this phenomena but I see this a lot and it really bothers me, where leaders get “too” inspired by management or process ideas from blogs.
It almost never works out and then the leaders blame teams for incorrectly following whatever the process was rather than accepting that almost every org is somewhat unique so adjustments to strategy need to always be made.
It almost never works out and then the leaders blame teams for incorrectly following whatever the process was rather than accepting that almost every org is somewhat unique so adjustments to strategy need to always be made.
Isn't this just plain cargo cult management? They read someone claiming some process is the best, proceed to monkey the process and then get surprised when the imagined results don't materialize.
I didn't have a word for it but now I do.
This sentence also describes the vast majority of startups - "It’s when we imitate behaviors without understanding how they work in the hope of achieving the same results. Naturally enough, it’s not something that works."
This sentence also describes the vast majority of startups - "It’s when we imitate behaviors without understanding how they work in the hope of achieving the same results. Naturally enough, it’s not something that works."
I remember when that goddamn Steve Jobs book came out. For a good while every founder and CEO suddenly thought they were some kind of product wizard and started trying to imitate Jobs’s hands-on micro-detailed product design meddling.
I'm starting to wonder whether you could reliably predict management behavior at large corporations by tracking the bestseller biographies that came out. Or being current with HBR.
This is a very interesting idea. Now I'm getting curious to see how this could be put into action.
Whilst probably not as beneficial as predicting behaviour, there could be a startup version to find the origins of buzzwords. Then try to predict how management will potentially push that company forward using these buzzwords or references to external sources.
Whilst probably not as beneficial as predicting behaviour, there could be a startup version to find the origins of buzzwords. Then try to predict how management will potentially push that company forward using these buzzwords or references to external sources.
Spot on, it's infuriating to deal with. There's a lot of time spent reading blogs and quoting some flavor of the month founder and then essentially trying to replicate as much as possible.
One of my consulting roles got terminated early (1 week before I was about to terminate it myself too) because a Growth Hacking agency with a loud speaking founder turned up, told the founder that my work was not good (an agency saying that there's problems across the board - who would have thought that huh?) only for them to be fired 1 month later and an outcry across social media for 'real growth hackers' and not promising the world from that same founder.
Bearing in mind this founder also had the latest Apple products because it was a must...
One of my consulting roles got terminated early (1 week before I was about to terminate it myself too) because a Growth Hacking agency with a loud speaking founder turned up, told the founder that my work was not good (an agency saying that there's problems across the board - who would have thought that huh?) only for them to be fired 1 month later and an outcry across social media for 'real growth hackers' and not promising the world from that same founder.
Bearing in mind this founder also had the latest Apple products because it was a must...
Survivorship bias clubbed with no true Scotsman?
> So much frustration was felt because a manager read a TechCrunch article about how X (with 100's of millions) did this and achieved something spectacular.
Because they did this and (coincidentally) achieved something spectacular :)
Because they did this and (coincidentally) achieved something spectacular :)
The book "Seeing like a State" expounds on this topic significantly, using scientific forestry as an example. The "recipe for failure" [0]:
1. Look at a complex and confusing reality
2. Fail to understand all the subtleties of how the complex reality works
3. Attribute that failure to the irrationality of what you are looking at, rather than your own limitations
4. Come up with an idealized blank-slate vision of what that reality ought to look like
5. Argue that the relative simplicity and platonic orderliness of the vision represents rationality
6. Use authoritarian power to impose that vision, by demolishing the old reality if necessary
7. Watch your rational Utopia fail horribly
[0] https://smus.com/books/seeing-like-a-state-by-james-scott/
1. Look at a complex and confusing reality
2. Fail to understand all the subtleties of how the complex reality works
3. Attribute that failure to the irrationality of what you are looking at, rather than your own limitations
4. Come up with an idealized blank-slate vision of what that reality ought to look like
5. Argue that the relative simplicity and platonic orderliness of the vision represents rationality
6. Use authoritarian power to impose that vision, by demolishing the old reality if necessary
7. Watch your rational Utopia fail horribly
[0] https://smus.com/books/seeing-like-a-state-by-james-scott/
> 5. Argue that the relative simplicity and platonic orderliness of the vision represents rationality
That's an elegant way to articulate a criticism I've had of many technologies that I have been unable to state so clearly. All those file formats, for example, that seek to replace XML. Think of JSON, YAML, and TOML. They all claim to be simple, but end up too simple in practice and cause just as many issues as they solve. Comments, schemas, and namespaces were there for a reason, they weren't just a "messy design" awaiting a visionary to clean up...
> 6. Use authoritarian power to impose that vision, by demolishing the old reality if necessary
Unfortunately, in the cloud era, we're all faced with trillion dollar behemoth organisations that wield absolute power over their dominions. Us peasants have no hope of changing the vision they choose to impose. Suddenly, the well-established and well-known stable world has become "legacy".
That's an elegant way to articulate a criticism I've had of many technologies that I have been unable to state so clearly. All those file formats, for example, that seek to replace XML. Think of JSON, YAML, and TOML. They all claim to be simple, but end up too simple in practice and cause just as many issues as they solve. Comments, schemas, and namespaces were there for a reason, they weren't just a "messy design" awaiting a visionary to clean up...
> 6. Use authoritarian power to impose that vision, by demolishing the old reality if necessary
Unfortunately, in the cloud era, we're all faced with trillion dollar behemoth organisations that wield absolute power over their dominions. Us peasants have no hope of changing the vision they choose to impose. Suddenly, the well-established and well-known stable world has become "legacy".
> Comments, schemas, and namespaces were there for a reason
Oh, surely. The one largest problem of XML is that it's highly redundant. Nobody complains about those, just that it's verbose, had to write, and hard to locate oneself on the hierarchy when reading. All of those are caused by the redundancy. (Oh, and there is the issue that DTDs have an incredibly complex language that nobody knows entirely...)
As a rule, if it's a good feature for a programming language, it's also a good feature for a textual data encoding language. Ditto for bad features.
Oh, surely. The one largest problem of XML is that it's highly redundant. Nobody complains about those, just that it's verbose, had to write, and hard to locate oneself on the hierarchy when reading. All of those are caused by the redundancy. (Oh, and there is the issue that DTDs have an incredibly complex language that nobody knows entirely...)
As a rule, if it's a good feature for a programming language, it's also a good feature for a textual data encoding language. Ditto for bad features.
> They all claim to be simple, but end up too simple in practice and cause just as many issues as they solve.
The number may be the same, but the set of issues is different, leaving developers the choice if they'd rather work around the lack of namespaces and comments in JSON or around the opaque meta definitions and annoying syntax of XML.
And the popular vote was pretty clear on which design is preferred.
The number may be the same, but the set of issues is different, leaving developers the choice if they'd rather work around the lack of namespaces and comments in JSON or around the opaque meta definitions and annoying syntax of XML.
And the popular vote was pretty clear on which design is preferred.
I am currently refining the metrics I want to present to my management. I work in information security so we do not exist until a issue arises.
I base the value of a metric presented to management on the "and so what?" basis. In other words: how it helps them to better run the business.
There is the "annual objectives" kind of report: how much we did and how good we are. Meaningless.
There is the "current events" one (nr of incidents, etc.). Meaningless.
There is the "these are systemic, company-wise issues I need you to arbitrate over", or "we need to do that, but it means changes and incomfort". Actual decision making data, the worst to present because they show problems and not celebration.
I guess that this is the case of these organizations that do not provide green money to the company (aka. "cost centers"). i hate these metrics.
(please note that there are metrics that are useful to *operational teams* so that they realize there is a trend, or that some performance upgrades will be needed, but they should stay there)
I base the value of a metric presented to management on the "and so what?" basis. In other words: how it helps them to better run the business.
There is the "annual objectives" kind of report: how much we did and how good we are. Meaningless.
There is the "current events" one (nr of incidents, etc.). Meaningless.
There is the "these are systemic, company-wise issues I need you to arbitrate over", or "we need to do that, but it means changes and incomfort". Actual decision making data, the worst to present because they show problems and not celebration.
I guess that this is the case of these organizations that do not provide green money to the company (aka. "cost centers"). i hate these metrics.
(please note that there are metrics that are useful to *operational teams* so that they realize there is a trend, or that some performance upgrades will be needed, but they should stay there)
Can't you present it as a risk / benefit analysis? Basically at the beginning of the quarter the open / larger risks were XYZ and now those risks have been mitigated or reduced.
Here’s how that goes before the quarter, by deciders less close to the situation on the ground, or whose own job performance evaluation depends on the situation not being as dire: “You’re being alarmist. Don’t ask us to deploy troops, it’s not like Americans are going to actually storm their own Capitol.”
And then after the quarter, “OK, Americans didn’t storm the Inauguration, but are you sure that’s because you so-called mitigated the risk with 20K troops? All this money, no attack, maybe we should spend that money on teachers instead of troops.”
In this space, lack of negative outcomes breeds contempt for the mitigation. (Hmmm...)
Interestingly, in the very long long run, that might even be right, as better education on civics is solving the problem at the left, earlier in the production line.
So now you have a new problem, which is to convince those deciders not close to the problem and disinterested in openly admitting gaps, that they need to both spend on the 20K troops now, and double down on teachers for a payoff 10 years from now, so that after that they don’t have to spend on the troops... best luck.
And then after the quarter, “OK, Americans didn’t storm the Inauguration, but are you sure that’s because you so-called mitigated the risk with 20K troops? All this money, no attack, maybe we should spend that money on teachers instead of troops.”
In this space, lack of negative outcomes breeds contempt for the mitigation. (Hmmm...)
Interestingly, in the very long long run, that might even be right, as better education on civics is solving the problem at the left, earlier in the production line.
So now you have a new problem, which is to convince those deciders not close to the problem and disinterested in openly admitting gaps, that they need to both spend on the 20K troops now, and double down on teachers for a payoff 10 years from now, so that after that they don’t have to spend on the troops... best luck.
Well, in reality this is what I present because this is what people usually present.
What I am trying to say that if I was management, I would be only interested to see things that are useful to the business.
If my business needs to be compliant with XXX, then a green status for XXX is what I need.
If there is a problem information security cannot handle because (whatever reason: no commitment form another organisation, no budget, ...) I want to know that.
Knowing that there are 7 attacks/sec or that there were 17 incidents would not help me.
This also means that I trust Information Security to provide me the right information and do their job correctly otherwise. This is usually checked by an anual external audits (at least for financial compliance).
What I do not want (this time in my real role in infosec) is to spend time providing meaningless data which management does not use anyway (but since "everyone" shows them, they want to see it too for a reason that escapes me)
What I am trying to say that if I was management, I would be only interested to see things that are useful to the business.
If my business needs to be compliant with XXX, then a green status for XXX is what I need.
If there is a problem information security cannot handle because (whatever reason: no commitment form another organisation, no budget, ...) I want to know that.
Knowing that there are 7 attacks/sec or that there were 17 incidents would not help me.
This also means that I trust Information Security to provide me the right information and do their job correctly otherwise. This is usually checked by an anual external audits (at least for financial compliance).
What I do not want (this time in my real role in infosec) is to spend time providing meaningless data which management does not use anyway (but since "everyone" shows them, they want to see it too for a reason that escapes me)
For those inclined, there's a nice article by Marilyn Strathern, The Tyranny of Transparency [0] that echoes the sentiments in this article. It builds upon Hari Tsoukas' The Tyranny of light: The temptations and the paradoxes of the information society [1], which I heartily recommend if interested in reflections on this topic.
To entice, here's a small excerpt from the abstract of [1]:
> The overabundance of information in late modernity makes the information society full of temptations. It tempts us into thinking that knowledge-as-information is objective and exists independently of human beings; that everything can be reduced into information; and that generating ever more amounts of information will increase the transparency of society and, thus, lead to the rational management of social problems. However, as argued in this paper, the information society is riddled with paradoxes that prevent it from satisfying the temptations it creates. More information may lead to less understanding; more information may undermine trust; and more information may make society less rationally governable.
[0] https://www.tandfonline.com/doi/abs/10.1080/713651562 (alternative: https://sci-hub.se/https://doi.org/10.1080/713651562)
[1] https://www.sciencedirect.com/science/article/abs/pii/S00163... (alternative: https://sci-hub.se/https://doi.org/10.1016/S0016-3287(97)000...)
To entice, here's a small excerpt from the abstract of [1]:
> The overabundance of information in late modernity makes the information society full of temptations. It tempts us into thinking that knowledge-as-information is objective and exists independently of human beings; that everything can be reduced into information; and that generating ever more amounts of information will increase the transparency of society and, thus, lead to the rational management of social problems. However, as argued in this paper, the information society is riddled with paradoxes that prevent it from satisfying the temptations it creates. More information may lead to less understanding; more information may undermine trust; and more information may make society less rationally governable.
[0] https://www.tandfonline.com/doi/abs/10.1080/713651562 (alternative: https://sci-hub.se/https://doi.org/10.1080/713651562)
[1] https://www.sciencedirect.com/science/article/abs/pii/S00163... (alternative: https://sci-hub.se/https://doi.org/10.1016/S0016-3287(97)000...)
Great article. My take on it:
- Metrics give everyone a false sense of objectivity. If you actually look at any stats in depth, you will end up making decisions about what to measure, and what the numbers mean. These words are actually the most important part of any study, but are always left out in favor of the graph. Want to know how many people died of covid? You need to consider things like population adjusted deaths. And adjustment for age. And a a whole host of other things to do with how the data was collected, and how it was presented. The economic side is even more messy. How much would GDP have been if we'd done this or that? There's no number that's the answer, but we do need numbers.
- You'll always care about more metrics than one. There's no generally objective way to decide how to weight those. You might have a utility function that decides this, thus reducing the problem to a simple optimization, but the guns/butter balance has still got to come from some kind of value that you express.
- The big question is rules vs discretion. You set up an elaborate measuring system (eg GDP) and you imagine that you have certain levers that you can move in response to whatever situation occurs. Part of this is that actors expect you to behave a certain way. But what happens if something you thought would work mechanically doesn't behave like you thought? Should you move your levers contrary to your previous commitment?
- Finally, there are the unmeasurable sensations. You have a team, it feels wrong somehow. You talk to people, get opinions, you get a feel for what is wrong. None of it is something you found in your LOC/user summary, or CI pipeline fails, or whatever.
- Metrics give everyone a false sense of objectivity. If you actually look at any stats in depth, you will end up making decisions about what to measure, and what the numbers mean. These words are actually the most important part of any study, but are always left out in favor of the graph. Want to know how many people died of covid? You need to consider things like population adjusted deaths. And adjustment for age. And a a whole host of other things to do with how the data was collected, and how it was presented. The economic side is even more messy. How much would GDP have been if we'd done this or that? There's no number that's the answer, but we do need numbers.
- You'll always care about more metrics than one. There's no generally objective way to decide how to weight those. You might have a utility function that decides this, thus reducing the problem to a simple optimization, but the guns/butter balance has still got to come from some kind of value that you express.
- The big question is rules vs discretion. You set up an elaborate measuring system (eg GDP) and you imagine that you have certain levers that you can move in response to whatever situation occurs. Part of this is that actors expect you to behave a certain way. But what happens if something you thought would work mechanically doesn't behave like you thought? Should you move your levers contrary to your previous commitment?
- Finally, there are the unmeasurable sensations. You have a team, it feels wrong somehow. You talk to people, get opinions, you get a feel for what is wrong. None of it is something you found in your LOC/user summary, or CI pipeline fails, or whatever.
"Management by reliance ONLY on metrics leads us astray"
Gut vs Metrics don't have to be either/or. Gut also has its own limitations - Daniel Kahnemann has documented this very well in Nobel Prize winning research. So what should one do? An organisation has to be self aware that there are no "perfect" metrics. Therefore, one should clearly state the limitations of a metric(s). Then, make a qualitative judgment based on the metrics and their limitations. You make the limitations, assumptions transparent and communicate the risks, uncertainties. Metrics are an input to decision making, they aren't decision makers themselves.
Gut vs Metrics don't have to be either/or. Gut also has its own limitations - Daniel Kahnemann has documented this very well in Nobel Prize winning research. So what should one do? An organisation has to be self aware that there are no "perfect" metrics. Therefore, one should clearly state the limitations of a metric(s). Then, make a qualitative judgment based on the metrics and their limitations. You make the limitations, assumptions transparent and communicate the risks, uncertainties. Metrics are an input to decision making, they aren't decision makers themselves.
Yeah, but it's about avoiding blame, not making the right decision.
Unfortunately good decision making is a rare skill, and management is a common profession, so stuff like this will continue to happen.
Note that the prerequisite for making good decisions is the space in which to make a bunch of decisions, some of which will inevitably be bad, so without a cultural acceptance of potential failure and the right attitude towards it, things will never improve.
Unfortunately good decision making is a rare skill, and management is a common profession, so stuff like this will continue to happen.
Note that the prerequisite for making good decisions is the space in which to make a bunch of decisions, some of which will inevitably be bad, so without a cultural acceptance of potential failure and the right attitude towards it, things will never improve.
I have noticed that labelling decisions as binary (right vs wrong, good vs bad) tends to create factions. I recommend turning decisions from binary into options/implications. A "bad" decision is an option with an implication assigned to it. If stakeholders are willing to agree with the implications, that's how it is.
Yeah, that's a problem. I kinda meant from the perspective of the person who made the decision in the future (i.e. normally me).
My favourite illustration of this is in The Wire. The city wants to crack down on crime and increases the target number of arrests. The police chief sends out the word and police officers begin making a bunch of spurious arrests. Do you blame the politicians for picking crappy metrics and for their poor understanding of human nature? Or do you blame the individuals who choose to exploit the metric out of self-interest? Plenty of blame to go around.
The Reply All episodes on the real life version of this are a really good listen if you haven't heard them already: https://gimletmedia.com/shows/reply-all/o2hx34/
(Short interview with one of the hosts on the episode: https://www.fastcompany.com/90249924/reply-all-podcast-nypd-...)
(Short interview with one of the hosts on the episode: https://www.fastcompany.com/90249924/reply-all-podcast-nypd-...)
The textbook example is when they wanted to reduce the rat population somewhere and paid people for (catching and) bringing in dead rats. Instead, people started to breed them and "sell" the dead specimens to the city.
It's the same result when one tries to run an economy through regulation, for the same reasons. A better way would be by incentives.
For example, there are regulations that 100 ppm of pollution X is illegal, while 99 ppm of X is legal. Emit 99 ppm, and you're good. Emit 100 ppm and your plant gets shut down. Even worse, you can emit 99 ppm for 10 hours and you're legal, but emit 100 ppm for 1 minute and you're illegal.
A far better approach is to tax total X emitted, not the rate, and not binary legal/illegal:
For example, there are regulations that 100 ppm of pollution X is illegal, while 99 ppm of X is legal. Emit 99 ppm, and you're good. Emit 100 ppm and your plant gets shut down. Even worse, you can emit 99 ppm for 10 hours and you're legal, but emit 100 ppm for 1 minute and you're illegal.
A far better approach is to tax total X emitted, not the rate, and not binary legal/illegal:
100 ppm for 1 minute times $1/ppm = $100 tax
99 ppm for 10 hours times $1/ppm = $59,400 taxA different approach is to incentivise behaviour sure, that doesn't mean it's inherently better.
A plant shutdown at 100ppm will likely result in running the plant at 90ppm to account for the risk of breaching the limit. But if I only pay an extra $100/min when run at 100ppm I'm better off running at 99ppm and just paying the extra tax when I occassionaly exceed 100ppm.
In your example, the tax incentivises you to pollute to the point you can afford. It might be cheaper to just pay the tax than actually make the required changes to your process.
However if the regulation sets a strict limit you don't have a choice, you have to make the changes necessary to comply with the regulations.
To the author's original point, it is important to look and what you are trying to achieve and why. In some cases tax incentives/penalties will make sense. In others strict limits/bans will be required.
A plant shutdown at 100ppm will likely result in running the plant at 90ppm to account for the risk of breaching the limit. But if I only pay an extra $100/min when run at 100ppm I'm better off running at 99ppm and just paying the extra tax when I occassionaly exceed 100ppm.
In your example, the tax incentivises you to pollute to the point you can afford. It might be cheaper to just pay the tax than actually make the required changes to your process.
However if the regulation sets a strict limit you don't have a choice, you have to make the changes necessary to comply with the regulations.
To the author's original point, it is important to look and what you are trying to achieve and why. In some cases tax incentives/penalties will make sense. In others strict limits/bans will be required.
> It might be cheaper to just pay the tax than actually make the required changes to your process.
The idea is to increase the tax rate so there is enough incentive for the total aggregate of pollution across all the producers to be reduced to an acceptable level.
The idea is to increase the tax rate so there is enough incentive for the total aggregate of pollution across all the producers to be reduced to an acceptable level.
At the very least the numbers of 99 ppm and 100ppm are measurement of the problem that one is trying to solve. The more typical braindead form of regulation would mandate one specific kind of mitigation regardless of the current level of pollution. While at the same time it is, of course, a pure coincidence that the more polluting company is friends with a politician while the less polluting company that now has to pay for less necessary or unnecessary mitigation is not friends with said politician.
> the numbers of 99 ppm and 100ppm are measurement of the problem that one is trying to solve
Not for air pollution. There the total emissions are of interest, not the rate. Car emissions, for example, are regulated for their rate of emission, not total emissions.
Not for air pollution. There the total emissions are of interest, not the rate. Car emissions, for example, are regulated for their rate of emission, not total emissions.
Even there, ppm is a suboptimal (maybe even terrible) measure. If I can find a way to pump additional air through the exhaust (increasing the denominator), I can lower my tailpipe ppm without lowering the rate of emitted pollutant (the numerator).
Of course, there are many ways to reduce the rate (as you mention), and of course, the government has made such modifications illegal.
My proposal is to measure the emission rate, multiply it by the mileage driven in the last year, multiply by the tax rate, and pay the tax to renew the license.
It's not perfect, but it's simple and a far better approximation than the existing brain-dead method. Instead of incentivizing drivers to just pass the test, they are incentivized to minimize the total pollution their driving emits.
My proposal is to measure the emission rate, multiply it by the mileage driven in the last year, multiply by the tax rate, and pay the tax to renew the license.
It's not perfect, but it's simple and a far better approximation than the existing brain-dead method. Instead of incentivizing drivers to just pass the test, they are incentivized to minimize the total pollution their driving emits.
In this case, “I” was intended to mean an automotive engineer working to comply with a government ppm-metric-driven regulation, not an end consumer, but my words were definitely ambiguous on that topic.
> would mandate one specific kind of mitigation
I agree that's bad. Far better to specify the goal, not the process.
I agree that's bad. Far better to specify the goal, not the process.
It's not the metrics per se - but the metrics not being totally aligned with the required outcome.
Good project management means being better at finding the right metrics - or at least recognising that our metrics aren't good enough, often because we haven't all of the information - why are we hiring a new developer, why is this piece of software being developed etc.
Good project management means being better at finding the right metrics - or at least recognising that our metrics aren't good enough, often because we haven't all of the information - why are we hiring a new developer, why is this piece of software being developed etc.
“We just need to find the appropriate XXX” is a good approach if no one gains from keeping a sub-optimal XXX (whatever it is).
A dumb but easy to reach metrics helps if you’re employee that wants money on next evaluation.
An ad targeted at the wrong people still brings money to the ad platform, helping find the good targeting mechanism is of low to negative ROI.
And so on. As long as it’s an adversarial situation you’ll never reach that appropriate target that needs to be shared among a number of people so they move in the same direction.
I think metrics are unavoidable for management, but they shouldn’t be on what you actually strive for, otherwise you’re just sending wishes to a monkey’s paw.
A dumb but easy to reach metrics helps if you’re employee that wants money on next evaluation.
An ad targeted at the wrong people still brings money to the ad platform, helping find the good targeting mechanism is of low to negative ROI.
And so on. As long as it’s an adversarial situation you’ll never reach that appropriate target that needs to be shared among a number of people so they move in the same direction.
I think metrics are unavoidable for management, but they shouldn’t be on what you actually strive for, otherwise you’re just sending wishes to a monkey’s paw.
The problems with metrics is that they indicate there is a single (multivariate) objective by running a company, running a country, living a life, running a whatever.
The thing is just that it is not the case. It in intricate networks of relations, aspirations, missions, ambitions, needs and so forth.
Metrics should underlie that fact, not overlie it. Ie. they can be used, just in moderation.
The thing is just that it is not the case. It in intricate networks of relations, aspirations, missions, ambitions, needs and so forth.
Metrics should underlie that fact, not overlie it. Ie. they can be used, just in moderation.
No matter which metrics you pick, as long as they are known, they will be gamed.
There’s usually no such thing as “the right metrics”. Metrics are helpful, yes, but they can only take you so far.
I've experienced this first hand so many times. It is so difficult to steer the boat when the management depends heavily on metrics they don't understand the need of. Just because others are doing it. And as the flow of pressure goes from top to bottom, engineers end up working towards achieving a metric which burns out the good ones and they leave.
I've spent sometimes hours in talking to clients on how they should first focus on metrics that help them ask the right questions.
This is prevalent everywhere, Scrum-masters holding meetings to discuss the scrum velocity.
I've spent sometimes hours in talking to clients on how they should first focus on metrics that help them ask the right questions.
This is prevalent everywhere, Scrum-masters holding meetings to discuss the scrum velocity.
While I agree with what I read as the premise of the article (that metrics for the sake of metrics is harmful and that qualitative data conversation is a great supplement to quantitative metrics) I wish the post was written with less hyperbole, especially the around big companies and politicians sections.
My take is that quantitative metrics are incredibly useful when paired with strong leadership, a clear mission, and vision statement that give human context to the metrics. That way whenever “the numbers” are brought up, the context and focus are around goals that really matter (are we creating a product that is meaningful and making people’s lives better?) versus cherry-picking and gaming the system to meet KPIs. Quantitative metrics are a dangerous tool since numbers are so easily digestible, so special care should always be given to understand exactly what the metrics are holding people accountable to.
An example that comes to mind in personal life is fitness. Weight is such an easy metric to get your hands on... doubly so since you can quickly judge weight by looking at yourself in the mirror. It’s really easy to tunnel on trying to lose weight! The activities (both physical and mental!) encouraged by fixation on weight can be incredibly destructive. If instead I first start with a strong vision statement that aligns with my values: My body is my most important possession; I want to keep it healthy so I can live a fulfilling life. In the context of that purpose weight is an important metric to keep in mind as a risk factor, but it is also clear that there are a multitude of other metrics that will allow me to measure, stretch, and grow towards a productive end.
My take is that quantitative metrics are incredibly useful when paired with strong leadership, a clear mission, and vision statement that give human context to the metrics. That way whenever “the numbers” are brought up, the context and focus are around goals that really matter (are we creating a product that is meaningful and making people’s lives better?) versus cherry-picking and gaming the system to meet KPIs. Quantitative metrics are a dangerous tool since numbers are so easily digestible, so special care should always be given to understand exactly what the metrics are holding people accountable to.
An example that comes to mind in personal life is fitness. Weight is such an easy metric to get your hands on... doubly so since you can quickly judge weight by looking at yourself in the mirror. It’s really easy to tunnel on trying to lose weight! The activities (both physical and mental!) encouraged by fixation on weight can be incredibly destructive. If instead I first start with a strong vision statement that aligns with my values: My body is my most important possession; I want to keep it healthy so I can live a fulfilling life. In the context of that purpose weight is an important metric to keep in mind as a risk factor, but it is also clear that there are a multitude of other metrics that will allow me to measure, stretch, and grow towards a productive end.
It comes down to what kind of people are in the positions. If they are kind of people that only care about numbers not amount of metrics will change that, if they are strong leaders with vision they usually have their custom ways of dealing with metrics.
I think the author of the article is trying to illustrate that the result of even a few metric-cheaters is that over time the entire organization or field is dominated by the metric-cheaters. There is just no way to always keep such people out of your organization once you cross a certain size threshold. So the logical result is that with time, everyone is forced to become more like the metric cheaters just to survive so they either leave or become metric cheaters themselves; in either case the product goes to shit as the creative problem solvers leave.
There’s a sweet spot that’s hard to hit. On one end you have people who ignore metrics. They get themselves into trouble breaking things without making the easy measurements to know they’re breaking them. They make obvious mistakes.
The next group makes subtle mistakes. They’re the ones who have learned how to use metrics and now they have a hammer, everything looks like a nail. I see this a lot in less senior data scientists. They know how to drive with data, and so everything must be done that way. But they make the mistakes this article gets at.
The group I like takes a middle ground approach. We recognize that certain network effects are so coupled that you can power maybe one proper experiment on it at a time, and even those experiments might take too long. Other experiments take too long simply by virtue that the thing you want to measure might be long term behavior. So you experiment on what you can, get everything you can by proxy, and make sure to at least measure health via the metrics that matter, but you don’t try to base every decision in something measurable simply because it’s measurable.
The next group makes subtle mistakes. They’re the ones who have learned how to use metrics and now they have a hammer, everything looks like a nail. I see this a lot in less senior data scientists. They know how to drive with data, and so everything must be done that way. But they make the mistakes this article gets at.
The group I like takes a middle ground approach. We recognize that certain network effects are so coupled that you can power maybe one proper experiment on it at a time, and even those experiments might take too long. Other experiments take too long simply by virtue that the thing you want to measure might be long term behavior. So you experiment on what you can, get everything you can by proxy, and make sure to at least measure health via the metrics that matter, but you don’t try to base every decision in something measurable simply because it’s measurable.
I agree. I think you need to be very careful about giving uni-directional advice like the OP, when in fact you are optimizing a U-shaped curve. Too little AND too much use of metrics is bad. The advice for a given company will depend where they are on that curve.
> A resume will never tell you if a person is genuinely interested in uncovering unknown truths about nature. But a 30-minute conversation will.
> knowing that nobody has been able to come up with a similar formula for interviewing programmers, where there are at least in theory easily measurable knowledge levels?
There is a technique that's successful and widely used. It's somewhat unfashionable to talk about it in today's political climate, it requires doing some actual work, and it isn't a "formula" - but it works.
The technique is exactly what TFA describes: talk to the candidate. Apply a domain-specific bozo filter (like FizzBuzz) if you must to narrow down the pool of candidates, but ultimately what you need to do is to get multiple people from the team for whom you're hiring to talk with the candidate for some hours, in a semi-structured way.
Cue in accusations of bias. Sure, it's biased, but what's the alternative? Pretending you can codify worker competency and fit as a set of cheaply testable metrics that are independent of company/team specifics? Such process doesn't exist, and will never exist, because the problem is AI-complete. Instead, why not trust that the people on your team are decent human beings looking for their own interest?
And in fact, a lot of companies do just that and happily chug along. But admitting to that doesn't make you an industry thought leader, nor can it be packaged into a service and sold.
There is a technique that's successful and widely used. It's somewhat unfashionable to talk about it in today's political climate, it requires doing some actual work, and it isn't a "formula" - but it works.
The technique is exactly what TFA describes: talk to the candidate. Apply a domain-specific bozo filter (like FizzBuzz) if you must to narrow down the pool of candidates, but ultimately what you need to do is to get multiple people from the team for whom you're hiring to talk with the candidate for some hours, in a semi-structured way.
Cue in accusations of bias. Sure, it's biased, but what's the alternative? Pretending you can codify worker competency and fit as a set of cheaply testable metrics that are independent of company/team specifics? Such process doesn't exist, and will never exist, because the problem is AI-complete. Instead, why not trust that the people on your team are decent human beings looking for their own interest?
And in fact, a lot of companies do just that and happily chug along. But admitting to that doesn't make you an industry thought leader, nor can it be packaged into a service and sold.
'OKRs before KPIs'as it says in our culture deck
https://citizenshareholders.com/culture
If achieving your KPIs compromises your Objectives and Key Requirements you have the wrong KPIs.
In the start-up/scale-up world this there can be enormous pressure to meet investor expectations on short term revenue/growth forecasts.
Part of the job of founders is to know their market, their company's unfair advantage.
'Don't be Bullied' should probably be the title of a blog post calling on founders to resist investor pressure to follow the norms in achieving early revenue, not because early revenue is a bad thing, far from it, it's amazing. But if the cost of generating that revenue puts OKRs at greater risk, its better to raise more money to extend your runway.
If you meet your OKRs, assuming you've chosen the right OKRs, your KPIs will follow.
In many ways this is a culture issue:
Culture trumps Strategy, OKRs trump KPIs
Provided you have the cash.
Citizen Shareholders is now looking for that cash. When we're asked why we have no revenue projected for 2 years, it's because being able to survive for 2 years without revenue will derisk us to the early B2B adopters.
Having that early revenue in the bank alongside those investor funds is a problem we can live with.
Just in time supply chains are so very 2019, we plan to make sure we have enough to succeed by, at times, having more than enough.
If you want to find out more: CitizenShareholders.com
Unfortunately many people miss the O in OKRs. The Objective is what really matters, but people tend to think it's all about the KRs - then you do end up with optimising for the metric.
Thank you!
Your post hit really hard, because a) Sole focus of my team IS MRR. b) I wrote something very similar. Sorry if my comment is a little maendering, I am a dad as of 7 days and what they tell you about sleep is no joke.
I want to plug in one of the less famous Drucker quotes:
> Working on the right things is what makes knowledge work effective. This is not capable of being measured by any of the yardsticks for manual work.
It's true that "What gets measured gets managed" by the converse is also true - if you focus on one metric only, the rest will fall into disarray.
It's good to keep your eyes at the gauges when you are flying, but just as the map is not the territory, the measurement is not the real thing.
People love metrics, because they give them an illusion of control over a complicated, real world, messy reality. The core of the problem is unfortunately human bias to simplify. We as a wider tech community are contributing a bit introducing yet-another-dashboard.
But as you rightly point, the metrics people pick are the ones that are easiest to measure. NOT the ones worth measuring. In fact I published a post quite similar to yours: https://piszek.com/2019/11/24/metrics/
To broaden the theme, I would even argue that Taleb is complaining about roughly a similar set of problems: Gaussian curve is an oversimplification as well and has caused a lot of suffering when misapplied.
Despite all of these caveats, there is a benefit to having a focus (like MRR you are mentioning). Having a singular focus can be transformative and can make one developer more effective than 30-people team with no real direction.
In that sense, there is benefit to metrics. As with all real-world things, I think the only solution is a dynamic equilibrium, where you change from one approach to the other to cover your bases.
As I wrote in the beginning of this comment, the Team I Lead (we build simple ecommerce tools) was tasked with maximizing MRR and the side-effect of having that focus was that we ignored the one-off sales features that our customers wanted desperately.
Your post hit really hard, because a) Sole focus of my team IS MRR. b) I wrote something very similar. Sorry if my comment is a little maendering, I am a dad as of 7 days and what they tell you about sleep is no joke.
I want to plug in one of the less famous Drucker quotes:
> Working on the right things is what makes knowledge work effective. This is not capable of being measured by any of the yardsticks for manual work.
It's true that "What gets measured gets managed" by the converse is also true - if you focus on one metric only, the rest will fall into disarray.
It's good to keep your eyes at the gauges when you are flying, but just as the map is not the territory, the measurement is not the real thing.
People love metrics, because they give them an illusion of control over a complicated, real world, messy reality. The core of the problem is unfortunately human bias to simplify. We as a wider tech community are contributing a bit introducing yet-another-dashboard.
But as you rightly point, the metrics people pick are the ones that are easiest to measure. NOT the ones worth measuring. In fact I published a post quite similar to yours: https://piszek.com/2019/11/24/metrics/
To broaden the theme, I would even argue that Taleb is complaining about roughly a similar set of problems: Gaussian curve is an oversimplification as well and has caused a lot of suffering when misapplied.
Despite all of these caveats, there is a benefit to having a focus (like MRR you are mentioning). Having a singular focus can be transformative and can make one developer more effective than 30-people team with no real direction.
In that sense, there is benefit to metrics. As with all real-world things, I think the only solution is a dynamic equilibrium, where you change from one approach to the other to cover your bases.
As I wrote in the beginning of this comment, the Team I Lead (we build simple ecommerce tools) was tasked with maximizing MRR and the side-effect of having that focus was that we ignored the one-off sales features that our customers wanted desperately.
>> Who else manages by metrics? Politicians!
This is completely false. Politics is the least metrics driven, mostly intention driven.
https://www.washingtontimes.com/news/2014/apr/25/sowell-good...
This is completely false. Politics is the least metrics driven, mostly intention driven.
https://www.washingtontimes.com/news/2014/apr/25/sowell-good...
I think you are probably right but the article you linked to doesn't demonstrate that.
> You know who manages by metrics? Big companies like Google, Amazon and LinkedIn.
And what do they have in common?
Their core product got notably worse over time.
It’s interesting he doesn’t mention Apple. I have no insight but from the outside of say they have very different things they optimize for than revenue. Money merely pours out of their customer obsession instead.
And what do they have in common?
Their core product got notably worse over time.
It’s interesting he doesn’t mention Apple. I have no insight but from the outside of say they have very different things they optimize for than revenue. Money merely pours out of their customer obsession instead.
As usual: If you can't measure what's important you declare important what you can measure. That wouldn't be so destructive if management at least was aware of that problem. But they aren't. They'll have their report of metrics which are green or red. Green is good. Red is bad. End of story.
All employees have radically different ideas about how to organise for success, ranging from free market capitalism to the government literally seizing control of this specific company. The skill range is from "needs constant supervision" to "good at a very limited range of activities". Any specific manager has 40 hours a week, which gets eaten up very quickly (10 reports = 4 hrs each to figure out if you like what they are doing assuming no actual work the manager needs to do or surprises that mean they can't work to schedule).
The issue isn't that anyone believes that metric-driven management is a good option, it is that at the scale of 2,000+ people there are not a lot of alternatives. I could easily design a system that works at large scale ... but only by the employees and owners being the same people. Apart from that, the erratic us-them nature of stockholders and employees makes all the good structures untenable.
Lay out an org chart for 2k people then ask how long the manager has to work out what their reports are doing. Expand the hierarchy far enough that everyone has time to do a good job and then ask what the odds are that all the people in positions of importance agree on how things should be done by whom. To settle the number of disputes that arise in a big tree-like org structure there isn't have much of a plan B apart from metrics.
It would be nice to see more governmental support for limited liability cooperative business models. The organisational opportunities that would open up could allow for so much. Not to mention maybe workers could earning more.
The issue isn't that anyone believes that metric-driven management is a good option, it is that at the scale of 2,000+ people there are not a lot of alternatives. I could easily design a system that works at large scale ... but only by the employees and owners being the same people. Apart from that, the erratic us-them nature of stockholders and employees makes all the good structures untenable.
Lay out an org chart for 2k people then ask how long the manager has to work out what their reports are doing. Expand the hierarchy far enough that everyone has time to do a good job and then ask what the odds are that all the people in positions of importance agree on how things should be done by whom. To settle the number of disputes that arise in a big tree-like org structure there isn't have much of a plan B apart from metrics.
It would be nice to see more governmental support for limited liability cooperative business models. The organisational opportunities that would open up could allow for so much. Not to mention maybe workers could earning more.
Could you give an example of what you mean in the last paragraph? What types of additional government support would help? (Are you talking legal, financing, taxation, reduced paperwork/overhead/fees or other?)
Seems like a thorough listing. Yes.
I'd like to see the major changes in taxation and legal reform to favour encourage bigger risk/reward sharing with workers and reductions in paperwork and regulation to make it easier to set up companies with a diverse range of small, actively involved stakeholders.
I think it is notable how well American tech companies did with large equity handouts to their early employees. Maybe the model starts there. I don't have details, I do have a problem - to get great results, large complex organisations need a different model than salaried workers.
I'd like to see the major changes in taxation and legal reform to favour encourage bigger risk/reward sharing with workers and reductions in paperwork and regulation to make it easier to set up companies with a diverse range of small, actively involved stakeholders.
I think it is notable how well American tech companies did with large equity handouts to their early employees. Maybe the model starts there. I don't have details, I do have a problem - to get great results, large complex organisations need a different model than salaried workers.
Limited liability worker cooperatives are very common in some businesses.
> You know who manages by metrics? Big companies like Google, Amazon and LinkedIn.
This isn’t true through. On this very site’s Startup School you will find lots of experienced entrepreneurs advocating for a metric-driven approach to growing your startup. Maybe not “one metric that matters” all the time, but Lean Startup is about measuring your company’s metrics and running experiments week-by-week to see what moves the needle.
Sure, slavish and myopic adherence to a bad metric will lead you astray. But management without metrics will lead you astray too.
This isn’t true through. On this very site’s Startup School you will find lots of experienced entrepreneurs advocating for a metric-driven approach to growing your startup. Maybe not “one metric that matters” all the time, but Lean Startup is about measuring your company’s metrics and running experiments week-by-week to see what moves the needle.
Sure, slavish and myopic adherence to a bad metric will lead you astray. But management without metrics will lead you astray too.
That's not true and true at the same time. There are myriad instances where management by metrics leads companies totally astray, bit it can easily argued that they are badly managed or have a bad strategy.
The core of any kind of measurement effort is a clearly defined objective that sits in balance with other objectives, and the important thing is these objectives are sometimes in conflict with each other, like finance and UX, measuring both gives the company a chance to find the right balance.
Eric Reis' "The Lean Startup" covers this topic in detail, and makes a useful distinction between "vanity" metrics and "actionable" metrics. Aside from the book, you can read a blog post by him on the subject of metrics here:
https://tim.blog/2009/05/19/vanity-metrics-vs-actionable-met...
By using metrics we make it a lower dimensional model, that can be managed at scale. However in real world, it often becomes a overfit one. In large scale execution, people only care what metric measures.
A talk vs metrics is just higher dimension vs lower one. The information within a talk is complex. Different people received vary information from the same talk. So it's hard to managed by scale.
So a metrics system is just a trade-off for its scale.
A talk vs metrics is just higher dimension vs lower one. The information within a talk is complex. Different people received vary information from the same talk. So it's hard to managed by scale.
So a metrics system is just a trade-off for its scale.
I find this to be very very rich.
A guy who lambasts metrics, but whose projects are all based on scraping metrics.
https://productexplorer.io/ https://newsletterspy.io/ https://gumspy.com/
A guy who lambasts metrics, but whose projects are all based on scraping metrics.
https://productexplorer.io/ https://newsletterspy.io/ https://gumspy.com/
I don’t think there is an alternative to management by measurement if scalable is what you’re trying to be.
That doesn’t mean you don’t have long term goals or nuanced objectives even if your measures are coarse, nor does it mean your measures stay the same.
I think optimising for measures focuses activity , which again doesn’t mean you have to optimise for the same thing all the time using the same measures.
That doesn’t mean you don’t have long term goals or nuanced objectives even if your measures are coarse, nor does it mean your measures stay the same.
I think optimising for measures focuses activity , which again doesn’t mean you have to optimise for the same thing all the time using the same measures.
The core question which should always be asked: when defining or setting a goal, what kind of incentives are you simultaneously creating?
I'm a but cynical about the recommendation of talking to people. The interviews then turn into its own optimisation game (I think politicians for instance do base their decision making on talking to people, hence lobbying). The same thing happens with job interviews (what is the most useful skill to getting a great SWE salary? Interviewing well).
Metrics in software is a bag of issues in most companies I see. But there ARE useful ways to use metrics that actually help (software). I did a talk about this at GOTO: Lies, Damned lies, and metrics.
https://www.youtube.com/watch?v=goihWvyqRow
The article fails to acknowledge that there is in fact two very distinct families of metrics.
1. Taylor metrics - The ones invented by management. 2. von Braun metrics - The ones invented by the guy that put men on the Moon. These are everywhere it really matters.
1. Taylor metrics - The ones invented by management. 2. von Braun metrics - The ones invented by the guy that put men on the Moon. These are everywhere it really matters.
Article shows companies like Google and Facebook. However doesn't talk about big companies managing in a different way. Why? They are all dead. Entropy and decay is part of product cycle. There is no way around it. You push them far away.
> Let’s say my goal this year is to get to 10k MRR.
Monthly Recurring Revenue, if anyone is wondering.
Monthly Recurring Revenue, if anyone is wondering.
Thanks! Otherwise it's a great blog post, but I did stumble over that unexplained acronym (and was too lazy to google it)
Thanks for the feedback. Just changed the first sentence.
The metrics being discussed here are largely quantitative. OTOH, a qualitative metric is better for measuring if your product is turning to shit over time. Both types are needed to manage big complex things.
I very much subscribe to combining this 'Common-sense filter' to metrics when choosing a course of action, and came up with a term for it: inform your gut.
That's a perfect way to phrase it. Thanks!
When the situation allows, you want to pick a metric that would not matter if it were gamed. It takes thoughtful leadership to arrive at it.
The conclusion of this article is basically "I don't understand how to use common sense and data at the same time, so I'll hark back to old-timey "gut instinct" and good-ol-fashioned selection bias hidden by my own lack of self-understanding to validate my ignorance, oh and here's some BS about lockdown policies."
What a joke, all of these posts have the same conclusions: don't deal in absolutes.
What a joke, all of these posts have the same conclusions: don't deal in absolutes.
> Their core product got notably worse over time.
> Google’s search results are dominated by ads
Googles core product are ads, not search engine!
> Google’s search results are dominated by ads
Googles core product are ads, not search engine!
But their ability to sell ads critically relies on their dominance in the search engine market, which is based on the quality of the search results.
There was a time before Google. There were other search engines with large market shares. Google completely destroyed them through superior search results.
An alternative search engine that actually works better as a search engine would be an existential threat to Google, and they know that. Because there is very little that prevents people from switching search engines - no network effects, nothing really except default settings and laziness.
There was a time before Google. There were other search engines with large market shares. Google completely destroyed them through superior search results.
An alternative search engine that actually works better as a search engine would be an existential threat to Google, and they know that. Because there is very little that prevents people from switching search engines - no network effects, nothing really except default settings and laziness.
I remember Altavista.
Adsense controls like 80% of ad market. Biggest thread to Google are walled gardens like Facebook or Amazon. Search engines like Bing and Duck are not even on their map.
> dominance ... which is based on the quality of the search results
Any source for that? Google search already provides inferior results in many areas, but it is still dominant. Google also pays billions to Apple and Mozilla to be default search engine.
And if threat emerges and gets like 30% of search market share, Google just dials down ads and make their search experience a bit less crappy.
Adsense controls like 80% of ad market. Biggest thread to Google are walled gardens like Facebook or Amazon. Search engines like Bing and Duck are not even on their map.
> dominance ... which is based on the quality of the search results
Any source for that? Google search already provides inferior results in many areas, but it is still dominant. Google also pays billions to Apple and Mozilla to be default search engine.
And if threat emerges and gets like 30% of search market share, Google just dials down ads and make their search experience a bit less crappy.
> Adsense controls like 80% of ad market.
Only about 14% of Google's advertising income is from AdSense, the rest is from ads on Google's own properties, and the vast majority of that is from search.
> Biggest thread to Google are walled gardens like Facebook or Amazon. Search engines like Bing and Duck are not even on their map.
I guarantee you they are. Facebook and Amazon are threats because they are established with massive userbases so anything they do can quickly have a big impact.
But I absolutely guarantee that competition from a better search engine is very much on Google's map as an existential threat, just not an immediate one. The know they cannot stagnate in that field and are continually investing into into research to stay at the bleeding edge.
> Any source for that? Google search already provides inferior results in many areas, but it is still dominant.
How do you even quantify search result quality? My personal experience from trying out other search engines is that Google is definitely superior, and better search results was most definitely what got them to dominance in the first place - Altavista and Yahoo and Excite weren't even in the same ballpark.
> Google also pays billions to Apple and Mozilla to be default search engine.
And that should tell you all you need to know about how essential it is for Google to stay the dominant search engine.
> And if threat emerges and gets like 30% of search market share, Google just dials down ads and make their search experience a bit less crappy.
At that point they would already be fighting desparately for their life, because "just dialing down ads" translates into an immediate, proprotional decrease of revenue from their (at that point already compromised) main cash cow. 30% less ads on 30% less searches would immediately mean Alphabet as a whole is deep in the red.
And if a competitor already managed to take substantial market share from them, making the experience "a bit less crappy" is hardly going to be enough to turn things around.
No, Google really must be (and is) much more proactive in this: keep investing in research, keep on the lookout for any company that looks like it's doing something that might someday in the future impact search engine technology, and just buy it.
Only about 14% of Google's advertising income is from AdSense, the rest is from ads on Google's own properties, and the vast majority of that is from search.
> Biggest thread to Google are walled gardens like Facebook or Amazon. Search engines like Bing and Duck are not even on their map.
I guarantee you they are. Facebook and Amazon are threats because they are established with massive userbases so anything they do can quickly have a big impact.
But I absolutely guarantee that competition from a better search engine is very much on Google's map as an existential threat, just not an immediate one. The know they cannot stagnate in that field and are continually investing into into research to stay at the bleeding edge.
> Any source for that? Google search already provides inferior results in many areas, but it is still dominant.
How do you even quantify search result quality? My personal experience from trying out other search engines is that Google is definitely superior, and better search results was most definitely what got them to dominance in the first place - Altavista and Yahoo and Excite weren't even in the same ballpark.
> Google also pays billions to Apple and Mozilla to be default search engine.
And that should tell you all you need to know about how essential it is for Google to stay the dominant search engine.
> And if threat emerges and gets like 30% of search market share, Google just dials down ads and make their search experience a bit less crappy.
At that point they would already be fighting desparately for their life, because "just dialing down ads" translates into an immediate, proprotional decrease of revenue from their (at that point already compromised) main cash cow. 30% less ads on 30% less searches would immediately mean Alphabet as a whole is deep in the red.
And if a competitor already managed to take substantial market share from them, making the experience "a bit less crappy" is hardly going to be enough to turn things around.
No, Google really must be (and is) much more proactive in this: keep investing in research, keep on the lookout for any company that looks like it's doing something that might someday in the future impact search engine technology, and just buy it.
Ok, you seem to have better numbers, I was just pulling numbers from top of my head.
My point is that search is not priority for google anymore. 15 years ago it won because it had clean simple UI. Now its bog down with ads. Duck and others offer much better experience.
My point is that search is not priority for google anymore. 15 years ago it won because it had clean simple UI. Now its bog down with ads. Duck and others offer much better experience.
While is is a good (and often discussed) point that the use of metrics causes distortions as the metrics themselves affect behavior, I am not sure that the author's proposed solution of relying instead on intuition is an improvement. While the data may introduce distortions, ignoring it in favor of "going with my gut" is how we get vaccine denialism, conspiracy theories, racism, and so many other negative outcomes.
I am also not convinced by the inclusion of the R number in the list of examples - it is unlike the others in an important way. The problem with metrics usually arises when the metric is a proxy for the goal. For example, test scores are not the same as knowledge, and so using test scores as a metric can result in teaching to the test, such that the metric is no longer a good proxy for knowledge. But the R number is not a proxy - it is the actual thing we want to change. (Of course, there are many knock-on issues, such as how we measure that number, which do introduce proxies. Discussion of that would be a much more interesting article.)
Which actually points towards the real solution, which is not to simply rely on gut instinct, but to be aware of what you are measuring, and how it relates to what you want to know.
I am also not convinced by the inclusion of the R number in the list of examples - it is unlike the others in an important way. The problem with metrics usually arises when the metric is a proxy for the goal. For example, test scores are not the same as knowledge, and so using test scores as a metric can result in teaching to the test, such that the metric is no longer a good proxy for knowledge. But the R number is not a proxy - it is the actual thing we want to change. (Of course, there are many knock-on issues, such as how we measure that number, which do introduce proxies. Discussion of that would be a much more interesting article.)
Which actually points towards the real solution, which is not to simply rely on gut instinct, but to be aware of what you are measuring, and how it relates to what you want to know.
”it was a good measurement until it became a target”
"What gets measured gets done" , unfortunately.
Much boils down to the question about the quality of the metrics. In my work they are just numbers that are manipulated to look good so that everyone gets their bonuses and the bosses are happy.
If anyone is interested in reading about this topic in terms of late stage capitalism I would highly recommend a book by Mark Fisher titled Capitalist Realism.
In it he talks about “Stalinist Capitalism” I.e. a capitalist society that is obsessed with metrics which optimize for arbitrary goals at the expense goals which are intangible or hard to measure. It focuses on the metric obsessed cultures that have developed within many private organizations.
Two examples come to mind, universities prioritizing publishing count vs publishing impact and lines of code written vs impact of code.
In it he talks about “Stalinist Capitalism” I.e. a capitalist society that is obsessed with metrics which optimize for arbitrary goals at the expense goals which are intangible or hard to measure. It focuses on the metric obsessed cultures that have developed within many private organizations.
Two examples come to mind, universities prioritizing publishing count vs publishing impact and lines of code written vs impact of code.
Also, this reminds me of stories about Soviet manufacturing quotas[1]:
> The Soviet Union was the largest producer of shoes in the world. It was turning out 800 million pairs of shoes a year–twice as many as Italy, three times as many as the United States, four times as many as China. Production amounted to more than three pairs of shoes per year for every Soviet man, woman, and child.
> The problem with shoes, it turned out, was not an absolute shortage. It was a far more subtle malfunction. The comfort, the fit, the design, and the size mix of Soviet shoes were so out of sync with what people needed and wanted that they were willing to stand in line for hours to buy the occasional pair, usually imported, that they liked.
Or even more ridiculous[2]:
> But probabably the most outrageous malfunction of the system was of a factory that fabricated metal products. It had a quota for scrap metal for recycling. The factory was operating more efficiently than expected so it was not generating as much scrap as expected. Since the factory was not meeting its scrap metal quota the higher authorities were going to fine it. So the factor personnel did what to them was rational. They took perfectly good zinc metal sheets and converted them into scrap to fulfill their scrap metal quota.
When looking at large organizations through the lens of information problems, one realizes that Amazon, Google, Walmart et al. all exist on continuum along with the Soviet Union or even the US Government.
[1]https://www.econlib.org/archives/2009/09/soviet_shoes.html
[2]https://www.sjsu.edu/faculty/watkins/stalinmodel.htm