Canary in a Coal Mine Metrics
Last week my IndustryWeek column titled Don’t Waste Your Metrics was released. It has generated a significant amount of discussion, as many companies find themselves in the situation of measuring far too many things, yet still not feeling like they know what is going on. I plan to build on this column, which is always quite space limited, with a series of blog posts on topics about metrics. This post is titled “The Canary in the Coal Mine.”
I’m not talking about the song by The Police; I’m talking about the analogy of how canaries were used in coal mines. They were used as sentinels to be early warning systems of toxic gases in the mine. The canaries would become sick from toxic fumes sooner than miners would notice them, and that would give them time to either escape or put on respirators.
When designed properly, metrics can accomplish the same thing.
Instead of trying to measure everything, choose something to measure that gives you an earlier indicator that something requires attention. That’s what metrics are for, after all. Here’s a couple of examples.
Many companies, particularly manufacturing companies, are employing the techniques of 5S, also known as workplace organization. I won’t go into all the reasons why; for that you can read one of my other posts or columns on the subject. But one critical aspect of 5S is the challenge of sustaining it. Audits because perhaps the most effective, albeit necessarily wasteful, methods to sustain 5S. For those in the trenches, having detailed checklists of what to check is very useful, because one thing out of place could create a significant problem down the path. But what if you’re the factory manager. Can you go through every shelf and toolbox looking for problems? Certainly not, at least not in a way that you can personally sustain. So what would be your canary in a coal mine?
In many areas, there is usually one spot that fills up with junk first. It’s usually a spot a little out of the way, and one that doesn’t have a purpose so clear that it keeps the junk out, much like the hall coat closet in many people’s homes. If our sustaining of 5S is going to fall apart, this is likely the first place it would show up. So instead of checking everything, just check this one spot. If you see it falling apart, you know some of the sustaining efforts are slipping and you can take corrective action before the rest of the area deteriorates. This one area is your canary in a coal mine.
What about customer satisfaction? Especially for companies with a broad customer base, deterioration of customer satisfaction doesn’t show up statistically very quickly. Further exacerbating the problem is that customers often have to experience the problem multiple times or at severe levels before they can be bothered with sharing the negative feedback. Therefore, by the time it shows up on your metrics dashboard, it may be very late.
Many companies have customers who are perhaps more sensitive. Perhaps they complain faster, or about more things, or about seemingly trivial things. These customers are not often talked about in favorable ways. But they can be used to serve you. If instead of looking at broad customer satisfaction data, use this one client, or group of customers, more closely. They will complain before other customers will. Yes, you will have to separate legitimate claims from the noise, but they are still an early warning system. If you insert a closer and more frequent look at that group of customers, it is your canary in a coal mine.
Most “canary in a coal mine” metrics are not comprehensive. They can’t be, by design. They are trying to predict a bigger problem, and they will sometimes get it wrong. Designing them will require both some creativity as well as deep understanding of cause-and-effect (a useful investment of time anyway). It will also require learning how to react, since there can be false signals.
But, wouldn’t you rather put on the respirator unnecessarily when the canary falls ill than end up experiencing the same condition yourself.
Reflection question: how do you design proactive canary-in-a-coal-mine metrics, and how do you manage the reaction to those metrics?