The misuse of metrics is the theme of "The Tyranny of Metrics" by Jerry Z. Muller

I was not surprised by historian Jerry Z. Muller‘s comments about “metric fixation” in his illuminating new book, The Tyranny of Metrics.

Some years ago the chairman and CEO of a Fortune 500 company remarked to me that nobody, not even Jack Welch, the then-idolized leader of General Electric, could possibly turn in solid revenue and profit increases steadily, quarter-after-quarter, year-after-year, through ups and downs in the market. It was clear to him that somebody was cooking the books at GE.

Estimated reading time: 5 minutes

I had been in business long enough then to understand how easy it is to shift sales and revenue from one quarter or year to the next and how other steps could be taken to fudge the numbers. (I was also aware of Welch’s reputation as “Neutron Jack” for his ruthless practice of “downsizing” to increase profits, a practice that can also be timed to give the appearance of steadily increasing profits.) It had been clear to me for many years that numbers can lie.

In his new book, Muller tackles our society’s obsession with metrics and accountability. However, “[t]his book is not about the evils of measuring,” he writes. “It is about the unintended consequences of trying to substitute standardized measures of performance for personal judgment based on experience. The problem is not measurement, but excessive measurement and inappropriate measurement—not metrics but metric fixation.” In just 200 pages, Muller assesses the use and misuse of metrics through case studies drawn from a wide range of fields: colleges and universities, K-12 education, medicine, policing, the military, business and finance, and philanthropy and foreign aid. The research he cites, and the examples he chooses, are compelling.

The Tyranny of Metrics by Jerry Z. Muller (2018) 227 pages ★★★★★

I’ve had my own experience with the misuse of metrics in both philanthropy and foreign aid. More than thirty years ago, when I was running an agency that raised money for not for profit organizations, I wrote and spoke to whoever would listen about the absurdity of measuring nonprofit performance on the basis of its fundraising costs. (You can find a condensed version of my thinking on that question at my post, Why financial ratios aren’t the way to evaluate nonprofit organizations.)

Years later, when I became involved in consulting with NGOs in developing countries, I saw for myself the folly of the metrics obsession that had seized hold of the international development community after Bill Gates began proselytizing on the subject. That fixation on the numbers forced far too many charities and government agencies to funnel money toward easily measured but trivial or even irrelevant programs while ignoring others that might actually have some positive impact in the field.

Muller sums up the problem nicely. “There are things that can be measured. There are things that are worth measuring. But what can be measured is not always what is worth measuring; what gets measured may have no relationship to what we really want to know. The costs of measuring may be greater than the benefits. The things that get measured may draw effort away from the things we really care about. And measurement may provide us with distorted knowledge—knowledge that seems solid but is actually deceptive.” Amen.

Examples you’re familiar with

Some of the examples Muller cites are familiar to the public at large. “Teaching to the test,” for example. The tendency of some surgeons to decline to operate in difficult cases because failure would lower their success ratios. And the tendency of police under pressure from politicians to make the numbers look better by classifying felonies as misdemeanors or altogether refusing to write up crimes. These are just a few of the many sad ways that our obsession with accountability distorts our understanding of the world around us.

Among the many unintended consequences of the misuse of metrics that Muller cites are the following:

  • Inducing people whose performance is measured to divert their efforts to what gets measured;
  • Promoting short-termism (as in Wall Street’s obsessive preoccupation with quarterly earnings reports at the expense of companies’ long-term health);
  • Discouraging innovation and risk-taking;
  • Sidetracking nonprofit staff members (or corporate employees, for that matter) from focusing on the mission that motivates them; and
  • Forcing employees to spend time logging data instead of doing their jobs (a requirement that was a major factor in convincing my brother to close his psychiatric practice many years ago).

Muller concludes The Tyranny of Metrics with a useful checklist of ten questions that any manager should ask when considering the application of metrics at work.

For related views, check out my reviews of two books about statistics: Charles Whelan’s Naked Statistics (How Netflix, Wall Street, and economists use and misuse statistics) and Nate Silver’s The Signal and the Noise (How experts make sense of the world (or don’t) with statistics). You might also enjoy what I wrote about Michael Lewis’ bestseller, The Undoing Project (Michael Lewis on the science of decision-making). And if Muller’s skepticism about the use of metrics in foreign aid raises any questions in your mind, read The Tyranny of Experts by William Easterly (Why economic development happens (or doesn’t)).

You might also enjoy Science explained in 10 excellent popular books (plus dozens of others)

If you enjoy reading nonfiction in general, you might also enjoy:

And you can always find my most popular reviews, and the most recent ones, on the Home Page.