This article originally appeared on LinkedIn.
The Wall Street Journal review of Tim Harford’s book The Data Detective quotes the economics maxim "When a measure becomes a target, it ceases to be a good measure."
One of the seemingly objective metrics that is often gamed is the ranking of colleges based on selectivity, as the review notes. The focus on this metric has led to efforts to inflate the pool of applications, so as to reject them in higher numbers. This is indeed perverse, as are score "begging" and other maneuvers aimed at upping customer service data including Net Promoter ScoreSM.
Statistical analysis and learning can’t happen until somebody starts collecting reliable data. That is why we started NPS® in the first place. But it is important to create a more consistent and transparent way of reporting numbers. Some companies have indeed begun to lay the groundwork for doing that with new tools and approaches. In the book, Harford makes the case that we all need to be smart consumers of data as well. We need to understand where data comes from, what sources were not included in its collection, potential biases—in both the data and ourselves—and how it fits historical trends.
As book reviewer David Shaywitz, a lecturer in the Department of Biomedical Informatics at Harvard Medical School, puts it, data can "illuminate and inform as well as distract and deceive. It’s often maddeningly hard to know the difference, but it would be unforgivable not to try."
I hope more of the companies that report their self-generated Net Promoter Scores to investors read Harford’s book. Same goes for executives who have unintentionally corrupted NPS by turning the measure into a target—and linking that target to employee bonuses.