Home » Technical Topics » Data Science

Simpson’s Paradox, the Bane of Clinical Trials

(This article is now a chapter of my github proto-book Bayesuvius)
Simpson’s paradox is a recurring nightmare for all statisticians overseeing a clinical trial for a medicine. It is possible that if they leave out a certain “confounding” variable from a study, the study’s conclusion on whether a medicine is effective or not, might be, without measuring that confounding variable, the opposite of what it would have been had that variable been measured. Statisticians have to enlist expert knowledge to assure themselves that no influential variables are left out.

Judea Pearl considers Simpson’s Paradox a fundamental problem which is greatly clarified by his theory of causal Bayesian networks (See references below).

Here is a simple example of Simpson’s Paradox (Or Simpson’s curse, or Simpson’s weed).

Some patients of both male and female genders are given a medicine or a placebo in a double blind study. Some recover from their ailment and others don’t. Let

r= recovered? No=0, Yes=1

t= took medicine? No=0, Yes=1

g= gender? Female=0, Male=1

The situation can be modeled by the Bayesian Network (bnet) in Fig.1
simpson-a

Figure 1

For this bnet, one has

P(r,g,t)=P(r|g,t)P(g|t)P(t)

Therefore,
P(r=1|t)= \sum_g P(r=1|t, g)P(g|t)=E_{\underline{g}|t} P(r=1|t, \underline{g})

whereE_{\underline{g}|t} is a conditional expected value (a kind of weighted average).

Supposeq_0, q_1 are non-negative real numbers. For the vector\vec{q}=(q_0, q_1):

Define a positive outcome (or success orq_t increasing with t ifq_0 \leq q_1.

Define a negative outcome (or failure orq_t decreasing with t ifq_0 \geq q_1.

simpson-vecs

Figure 2

It is possible (see Fig. 2 for a graphical explanation of how) to find perverse cases in whichP(r=1|t, g=0) andP(r=1|t, g=1) increase with t butP(r=1|t) decreases with t. So it is possible to conclude that the medicine is a success for each of the two g populations considered separately, yet the medicine is a failure when both populations are “amalgamated”. The lesson is that a “trend reversal” is possible upon amalgamation. The sign of the outcomes is not necessarily preserved when we do a weighted average of typeE_{\underline{g}|t}.E_{\underline{g}|t} is an expected value on the “confounding” random variable g conditioned on the root random variable t.

simpson-b

Figure 3

Footnote: Sometimes the bnet Fig.3 is used to explain Simpson’s paradox instead of the bnet Fig.1. But those two bnets are equivalent because

P(g|t)P(t)=P(g,t)=P(t|g)P(g)

References

  • “Understanding Simposon’s Paradox”, by Judea Pearl
  • Simpson’s Paradox: The riddle that would not die. (Comments on four recent papers), by Judea Pearl
  • “Simpson’s Paradox and the implications for medical trials’, by
    Norman Fenton, Martin Neil, Anthony Constantinou,

Leave a Reply

Your email address will not be published. Required fields are marked *