Subscribe to DSC Newsletter

Randomization, including A/B testing, often fails...for a variety of reasons.  When that happens you will likely find yourself in a situation where the treatment and control groups differ significantly.

First thing to remember is that some differences can occur in studies that are unavoidable.  Are the differences practically significant?  Are they statistically significant?

If differences exist that are significant then some options include:

-Prior to intervention, some statisticians recommend re-randomizing

- Can estimate the treatment effect conditional on the un-balanced covariates.

- Can adjust statistically for differences in the groups: for example in a regression model including the factors that are different in the model can reduce some of the bias in the estimate

- Can use quasi-experimental methods to improve the causal inference  (propensity methods, instrument variable, interrupted time series, etc.)

Having data that is not randomized makes inference more difficult this is a challenge that can be addressed with some straightforward analysis and data modeling.

#statistics #datascience #randomization #programevaluation

Views: 64

Tags: #statistics #datascience #randomization #programevaluation


You need to be a member of Data Science Central to add comments!

Join Data Science Central


  • Add Videos
  • View All

© 2020   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service