Thursday, March 15, 2012

Causes in Organizations: Learning From Samples of One (or Fewer)

Today I got the news that Judea Pearl had received the Turing Award from the Association of Computing Machinery. It made me look at my bookshelf where his book is sitting, a hefty volume where the math symbols crawl into the text of Chapter 1 and keep on coming. As early as chapter 3 there are important lessons, because it deals with how we know that a statement about cause and effect is true. The lessons are important because everyone who wants to accomplish something needs to have ideas about cause and effect, or a way of finding out links from cause to effect. In organizations we have a special need because we try to reliably and repeatedly accomplish things.

Thankfully we don't need to learn everything on our own. From an early age we are taught what actions are rewarding and what actions are not, without any need for experimentation. As a result, children know that eating vegetables has benefits beyond the immediate reward of getting dessert, and that unscrewing a light bulb and inserting their fingers will unpleasant. We maintain the habit of learning about cause and effect in adulthood because we are quite willing to ask when in doubt and to accept answers of those more experienced (or simply older). It is a highly effective habit when the consequences are subtle (health from vitamins) or are best avoided (electrical shocks).

Talking about cause and effect is not really a way of proving it. Knowing about causes and effects involve comparison of the effects in situations with the cause present and absent. More is involved too, but this one condition is enough to identify the problem. We usually try to explain anything unusual that we experience, which means that we identify the effect first, and then we look at anything that can be associated with it (preferably something unusual) and label it as a likely cause. Do we go through the experiment of removing the cause to see what happens? Scientists do, but in daily life we do not. Or, we do it as a mental exercise of thinking about how things could have been different. Those mental exercises are useful when we become the experienced people that others ask, because we can tell a complete story of cause and effect even though we only saw the effect and guessed a cause.

In organizations, this way of learning has interesting effects. Typically people notice the same unusual effect and start thinking and talking about it. They try out their proposed stories in conversations, and hold on to the stories that match those of others. If the effect they try to explain is a big event for the organization, these stories become part of the folklore of the organization. The accident that happened, the major client account that was won, the new product that became a big hit are effects that create stories. Because these stories are about causes and effects, others later use them to guide their own actions.

The problem with all this is that Chapter 3 of the big book of causality says that you can't really know that the proposed cause produced the effect unless you tried again without the cause, and saw how the effect changed. It would be difficult to run such an experiment even if the organization tried, because conditions have changed in the meantime. Usually organizations don't even try to run experiments. A big event happened, it was explained, and now the organization has a story and a nice new rule for action. Until it no longer works, of course, a new event occurs, and new explanations and rules are added.

March, J. G., Sproull, L. S., & Tamuz, M. 1991. Learning from samples of one or fewer. Organization Science, 2: 1-13.
Pearl, J. 2000. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University Press.