Metrics: when numbers change a company’s identity

In the video, in Italian, I start from a line by the sociologist William Bruce Cameron and get to creators, whom I follow the way I follow a company: when metrics become the purpose, they change whoever chases them. Here I tell the story of that line and of an idea that economists, sociologists and anthropologists have discovered more than once, and why it concerns every business.

Video transcript (translated from Italian)

William Cameron, a sociologist, wrote something in ’63 that I have always liked and that has stayed with me: “Not everything that can be counted counts, and not everything that counts can be counted.”

I often also work with people who make content online, who do online education for a living, influencers, who are actually companies too, and like every company they need a strategy behind them.

The problem, though, and this is where the reflection starts, is that strategy, too often and especially in this field of communication and education, gets replaced by an obsession with metrics, which go from being tools to being the only compass.

When metrics drive behavior, and this is true for influencers but also for every traditional company, they end up changing, transforming, shaping identity.

And that is the serious problem.

The model that worked because it was authentic collapses.

You have to understand one very simple thing: something that performs well is not necessarily something that creates value, something that positions you.

I said this a few years ago in a TEDx talk I gave: metrics are one of the parameters, not the parameter. If you optimize only what is easy to count, you risk losing exactly what counts.

A line written against the temptation of computers

In 1963 William Bruce Cameron, a professor of sociology, published an introductory book with an almost disarming title, Informal Sociology. On page 13, talking about the data his profession needs, he wrote: “It would be nice if all of the data which sociologists require could be enumerated because then we could run them through IBM machines and draw charts as the economists do. However, not everything that can be counted counts, and not everything that counts can be counted.”

A metric that becomes a goal changes the behavior of whoever chases it and, in the long run, their identity.

So the line I quote in the video was born with a precise irony. Those were the years when computers were entering universities, and a sociologist was reminding his colleagues that the most interesting part of what they studied would never fit on punch cards.

Since then the line has had a second life under another name. It circulates attributed to Albert Einstein, but according to the research of Quote Investigator the first link to Einstein appears only in 1986, in a book that actually attributed it to someone else. That is one more reason I like to quote it with the right name: a line about the difference between what is counted and what counts, circulating under the wrong name because the famous name gets more clicks, is almost a demonstration of its own point.

When a measure becomes a target, it stops measuring

The idea that a measure used as a target goes bad has been discovered at least three times, in three different disciplines. In July 1975, at a conference of the Reserve Bank of Australia in Sydney, the English economist Charles Goodhart observed that any statistical regularity tends to collapse once it is used as a lever of control. He was talking about British monetary policy, and about indicators that stopped behaving as before as soon as the central bank chose them as targets. The principle took his name: Goodhart’s law.

In 1976 the social psychologist Donald T. Campbell reached the same point starting from public policy. His version is harsher: the more a social indicator is used to make decisions, the more it is exposed to pressures that corrupt it, and the more it tends to distort the processes it is meant to monitor. Campbell counted it among his “pessimistic” laws.

The version most often quoted today is an anthropologist’s. In 1997 Marilyn Strathern, studying the assessment of British universities, wrote that when a measure becomes a target, it ceases to be a good measure. It is often attributed to Goodhart; the phrasing is hers, and she was drawing on a colleague who called it, precisely, Goodhart’s law.

In all three cases the mechanism is the same. As long as a number observes an activity, it describes it. When it becomes the reward, people learn to move the number, because that is easier than improving the activity.

The surgeons who avoided the sickest patients

The historian Jerry Z. Muller collected dozens of cases of this kind in The Tyranny of Metrics (2018), from schools to the police, from medicine to business. One of the clearest concerns surgery: when surgeons are publicly rated on their patients’ mortality, some stop operating on the most complex cases and choose the ones that improve their number.

This is not a suspicion. A study by David Dranove and colleagues, published in 2003 in the Journal of Political Economy, examined the report cards for heart surgeons introduced in New York and Pennsylvania. They led doctors and hospitals to select patients, and they worsened outcomes, especially for the sickest. The published number improved while care got worse.

With creators and educators something less dramatic but very similar happens. An educator starts publishing to say something; after a few months, looking at the statistics, they publish what the statistics reward. Each choice, taken alone, is reasonable. Put in a row, they change the tone, the topics and the audience, and the model that worked because it was authentic stops working just as the numbers grow.

Content or an initiative that does well in the numbers is not necessarily what creates value or positions a company.

Companies do the same with other indicators: leads generated by the website, deals closed by each salesperson, the customer satisfaction score. I talked about this back in 2022 at TEDxMacerata, when I said that metrics, born to help us find our way, had become the measure of value. It is also the subject of a book I reviewed, The Content Trap by Bharat Anand: in digital markets value lies in the connections between people and products, and those who produce content chasing the numbers lose sight of them.

What to count is decided by whoever runs the company

From all this one might conclude that we should stop measuring. That would be the wrong conclusion, and Muller does not draw it. The target of his book is metric fixation: the belief that standardized indicators can replace the judgment of those who know the work, and that the best way to motivate people is to reward or punish them on their numbers. Measurement used by the people who do the work, to see where to look, remains valuable.

For a company, the delicate point is the order of the decisions. First you decide what the company wants to be and what it will not do; then you choose the few numbers that tell you whether you are getting there. Done the other way around, the numbers decide the identity. It is the path I see most often, in companies and among creators, and putting it back in the right order is the first thing I do when I work on identity and positioning.

Metrics are one of the criteria for deciding, not the criterion: what is not counted must still be judged, with written rules.

In practice, written rules are a few sentences that say what the company will not do to improve a number. A company can decide, for example, that it will not cut prices to increase website leads, or that a salesperson will not be rewarded for deals with customers it does not want. A creator can decide which topics they will not write about, even if they perform. These are constraints no dashboard suggests, and they are precisely what keeps identity safe from the numbers.

Cameron, in the end, asked his colleagues for just one thing: to count everything that could be counted, without ever stopping looking at the rest.

What to do and what not to do

• Write down first what the company is and what it does not do, and only then choose the indicators that track it.
• Keep few indicators tied to rewards and incentives, and change them when people learn to move them.
• Read every number together with the judgment of people who know the work: the number tells you where to look, not what to decide.
• Do not judge a piece of content, a campaign or a colleague only by what the audience clicks.
• Do not treat an improving indicator as proof that the company is going in the chosen direction.

Sources

• William Bruce Cameron, Informal Sociology: A Casual Introduction to Sociological Thinking, Random House, 1963, p. 13. On the attribution to Einstein: Quote Investigator, 2010. https://quoteinvestigator.com/2010/05/26/everything-counts-einstein/
• Charles Goodhart, “Problems of Monetary Management: The U.K. Experience,” in Papers in Monetary Economics, vol. I, Reserve Bank of Australia, 1976 (conference of July 1975). https://www.rba.gov.au/publications/rdp/1990/9013/conference-volumes.html
• Donald T. Campbell, “Assessing the Impact of Planned Social Change,” Evaluation and Program Planning, 2 (1), 1979, pp. 67–90 (first version: Dartmouth College, 1976). https://doi.org/10.1016/0149-7189(79)90048-X
• Marilyn Strathern, “‘Improving ratings’: audit in the British University system,” European Review, 5 (3), 1997, pp. 305–321.
• Jerry Z. Muller, The Tyranny of Metrics, Princeton University Press, 2018 (Italian edition: Contro i numeri, translated by Alessia Cantagalli, Luiss University Press, 2019). https://luissuniversitypress.it/pubblicazioni/contro-i-numeri/
• David Dranove, Daniel Kessler, Mark McClellan, Mark Satterthwaite, “Is More Information Better? The Effects of ‘Report Cards’ on Health Care Providers,” Journal of Political Economy, 111 (3), 2003, pp. 555–588. https://doi.org/10.1086/374180