Risk and the Devil’s Advocate

Despite its venerable tradition and being an idiomatic English term, devil’s advocacy is underutilized in decision making. I believe this arises, in large part, from human nature. Few people like having their pet ideas rigorously challenged. Raising a few minor points is often tolerated and even encouraged in the guise of "free and open discussion", but often a person who persists in pointing out flaws in someone else’s thinking may get labeled as "not a team player" or even a "trouble maker". This encourages, even almost defines, "groupthink". The more important the emperor the less likely he is to enjoy having someone note that his ideas, if not his body, have no clothes. People seldom get bonuses for being seen as a pain-in-the-ass. The problem is that on the surface a true devil’s advocate is difficult to distinguish from a mere pain-in-the-ass. Thus people say, "Let me play devil’s advocate" but are not inclined to work at being devil’s advocate.

Even in risk analysis, which is supposedly a quantitative probabilistic science, what is purported to be careful analysis is too often a montage of vague guesstimates inserted into a framework of little analytical value (see The Failure of Risk Management: Why It's Broken and How to Fix It by Douglas Hubbard).

My Four Cs of Devil’s Advocacy

I use a framework for devil’s advocacy that I call the Four Cs: Challenge, Check, Confirm, and Care.

Challenge the assumptions.

Everyone’s thinking about a subject is based on assumptions. Most people do not take time even to enumerate, let alone examine, the relevant assumptions. Some assumptions are so long and deeply held that they become an unconscious part of our beliefs, so engrained in our thinking that we treat them like unquestioned facts of nature even when they are not. Even seemingly obvious assumptions may not necessarily be true, like the parallel axiom of Euclidean geometry. Challenging these can break down barriers to entirely new ways of thinking.

If you do nothing more than force people to state and evaluate their assumptions, you will have made a significant contribution.

Check the data.

Most people’s "data" is meager, gathered with dubious methodology, or just non-existent. Even when there seems to be adequate data, it is often biased or under-sampled in important portions of the distribution like the tails. There are Black Swans, events which are so far out of the experience of people as to be unthinkable. Granted by definition true Black Swans are beyond any expectation or at least so remote that no one will give serious thought to including the risks they represent, but most people underestimate even the probability of rare but conceivable events. Also the analysis of the data is always open to question or at least reinterpretation. People with limited experience are prone to applying statistical methods which are not valid for the data they have, either because they misunderstand the data or methods or they fail to verify that the data does not violate the assumptions of the methods.

Confirm the reasoning.

Relatively few people apply rigorous logic to decision making. Even when they think they are, frequently logical and cognitive fallacies take the place of valid syllogisms.

Care, but not that much.

Noted negotiator Herb Cohen applies this to reaching a deal. You should want to reach a deal but not so badly that you’re willing to accept a bad deal instead of walking away. In the context of devil’s advocacy, remember you are not offering an alternative model of how things are but merely trying, if possible, to poke enough holes in the model being discussed that it sinks. You don’t have a dog in the fight, so care about doing a good job but don’t feel you have to "win" every point. In the end your success is an improved decision making process and, hopefully, good decisions.

Home | Risk Defined | Weather & Climate Risks | News & Views | Credentials