Beware of vanity metrics

Follow three A's of metrics - actionable, accessible, auditable

The next article in my series on entrepreneurship for Harvard Business Review is live today. Once again, we revisit the topic of Actionable metrics and their nemesis: Vanity metrics. Any entrepreneur with a decent reality distortion field can find metrics that make it look like they're being successful. This "success theater" is occasionally useful for PR or getting through a tough board meeting. But it's lethal when we start to run the company using vanity metrics as a guide. But how do we know which metrics to look at? How do we know how much energy to invest in analytics vs. doing real work? And how do we get the whole team to take action that benefits real people when all they're looking at is abstract numbers on a screen? Remember "metrics are people, too." For more, read on...

Entrepreneurs: Beware of Vanity Metrics - The Conversation - Harvard Business Review:

The idea is simple. Establish baseline metrics by building the minimum viable product — the minimum required to measure the response of early adopters. Then, in each development cycle, use the insights gained by studying customers to make improvements. This is the source of validated learning — proof that the customer insights translate into tangible metrics improvements.

But this leaves a very difficult problem still to be solved: How do we know that these changes are what actually effect change in the metrics that we're observing?

This is the curse of vanity metrics, numbers which look good on paper but aren't action oriented: website hits, message volume, or 'billions and billions served.' They look great in a press release, but what do they accomplish?

Consider a scenario where a team makes a product change, and the very next month page views go up. As humans, we're hard-wired to infer causality from correlation: when the numbers go up, we tend to take credit. But when the numbers go down, we tend to blame someone or something else. Worse yet, different team members tend to attribute positive changes to whatever project they were working on at the time (but not negative changes, of course). As a result, different parts of the team are constantly "learning" in their own private reality. When those teams face difficult choices, it's incredibly hard for them to come together and make an informed, fact-based decision.

To avoid falling into this trap, I recommend you follow the three A's of metrics. All metrics should be actionable, accessible, and auditable...

(Image source: completeinnovator.files)