Subscribe to DSC Newsletter

Hi all,

What would you do if?

Bayesian Googe Optimize says probability to be best for Original is 64% (sales CR) and Frequentists's calculation says Original is higher that the Test by 10% (sales CR)  but the difference is not statistically significant or not enough sample size.
Meanwhile, Test version shows statistical significance in Add to Cart CR only.

Sample size is 200K. We cannot run the test more. Original is a delayed CTA page.

Would you stick with the Original hopping that it will indeed provide 10% boost in sales (64% probability to be best, 10% higher CR but not significant)? Or would you stick with Test version that has (15% higher CR in add to cart and significant)?

Views: 400

Reply to This

Replies to This Discussion

I suggest doing a third, separate analysis (hand-made) on the data that you gathered. There are many algorithms and options to perform A/B testing. Check the section on A/B testing in my book (here) page 259. 



  • Add Videos
  • View All

© 2020   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service