A Tail of 3 Models - The Story of Goodness of Fit with Binary Classification

Before you select the best model based on your favorite goodness of fit statistic – Mean Squared Error, Gini, K-S, AUC, or misclassification rate – STOP!  Model performance metrics are not a one size fits all measure.  As an analyst, selecting the right performance metric might mean the difference between having an exceptionally good result, and having no result.   

The classic example:  There is only a 3% prevalence of the event of interest in my data. I can build a model that is 97% accurate (3% error rate) that NEVER detects the event of interest!    In fact, I don’t even need to build a model to get this result – I can just guess “No” 100% of the time. 

CHeck out my blog post at: https://communities.sas.com/docs/DOC-2501



Views: 406

Tags: data, fit, goodness, logistic, mining, of


You need to be a member of Data Science Central to add comments!

Join Data Science Central

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service