Valid or reliable – is your culture right?
This week I’ve been writing about the tradeoff between measuring things in a valid way or a reliable means. Two posts published so far are:
How does this impact culture?
Culture is one of the most vital yet most difficult to measure dimensions in our business. Culture is best defined as the shared set of beliefs or principles and their corresponding behaviors held within an organization.
You can’t measure thinking, or observe it. The best we can observe is behaviors. Here’s how some work on this.
One is the cultural survey. We ask people what they are thinking, and we ask them how people behave around them. Many organizations develop their own questions, or they leverage something like Gallup’s 12 questions for engagement. A survey fits the reliable dimension. However, the validity is very low. Some of the best data perhaps comes from those that don’t even participate. Also, there is often a gap between what people say they do, and what they actually do. When people take the time to respond to a survey, they are often in the best of minds, not in the middle of a firefight when the behaviors matter most. So, it works, but we must be careful as the validity is low.
Another means is to focus on observing behaviors. This is one I prefer. It is mostly valid, because it is based on actual behaviors, not imagined or espoused ones. It loses some validity however as observing a behavior today is no guarantee that it will be the same behavior tomorrow. Where this approach really suffers is it’s reliability. Who does the observing makes a big difference. The conditions and environment in which we’re observing aren’t always consistent. And without a large sample size, it is more difficult to tell if the observed behavior is based on compliance or based on true thinking.
The point is, there is no one best way to measure culture. The reliability and validity challenges are significant. We should be very aware of the shortcomings of any methods we choose. The best strategy is to use many methods in concert together, and not be too beholden to the outputs of any one of them.