In the last part of the tutorial we introduced the basic qualitative model quality indicators. Let us recall them now:

- TP – True Positive – the number of observations correctly assigned to the positive class

Example: the model’s predictions are correct and resigning customers have been assigned to the class of „disloyal” customers - TN – True Negative – the number of observations correctly assigned to the negative class

Example: the model’s predictions are correct and customers who continue using the service have been assigned to the class of „loyal” customers. - FP – False Positive – the number of observations assigned by the model to the positive class, which in reality belong to the negative class.

Example: unfortunately the model is not perfect and made a mistake: some customers, who continue using the service have been assigned to the class of „disloyal” customers. - FN – False Negative – the number of observations assigned by the model to the negative class, which in reality belong to the positive class.

Example: unfortunately the model is not perfect and made a mistake: some churning customers have been assigned to the class of „loyal” customers.

We will now discuss derived variants of these indicators.

- TPR (True Positive Rate) – reflects the classifier’s ability to detect members of the positive class (pathological state)

TPR = TP / (TP + FN)

- TNR (True Negative Rate) – reflects the classifier’s ability to detect members of the negative class (normal state)

TNR = TN / (TN + FP)

- FPR (False Positive Rate) – reflects the frequency with which the classifier makes a mistake by classifying normal state as pathological

FPR = FP / (FP + TN)

- FNR (False Negative Rate) – reflects the frequency with which the classifier makes a mistake by classifying pathological state as normal

FNR = FN / (FN + TP)

- SE (sensitivity) – reflects the classifier’s ability to detect members of the positive class (pathological state)

SE = TP / (TP + FN)

- SP (specificity) – reflects the classifier’s ability to detect members of the negative class (normal state)

SP = TN / (TN + FP)

Note that the following equalities hold:

SE = TPR 1 – SE = FNR

SP = TNR 1-SP = FPR

- ACC (Total Accuracy) – reflects the classifier’s overall prediction correctness, i.e. the probability of making the correct prediction, equal to the ration of the number of correct decision to the total number of decisions

ACC = (TP + TN) / (TP + TN + FP + FN)

There are numerous measures of classifier quality and it is necessary to select the one which is appropriate for the situation at hand.

**Example**

During churn analysis we have two classes in the data: churn and no churn. There are 100 observations in the data and

- 90 observation belong to the class labeled as „0” (the positive class), which consists of customers who decided to stop using our services or products,
- 10 observation belong to the class labeled as „1” (the negative class), which consists of customers who continue to use our services or products.

For a certain classifier we obtain:

TP = 85 (85 churning customers have been classified as)

TN = 1 (1 not churning customer has been classified as such)

FP = 9 (9 not churning customers have been classified as churning)

FN = 5 (5 churning customers have been classified as not churning)

The sum of the above indicators equals the number of observations (customers):

TP + TN + FP + FN = 100

We now calculate the total accuracy of the classifier:

ACC = (85+1)/100 = 0,86

86 cases out of 100 has been correctly classified, which is not a bad result at all.

However, let us take a look at another indicator, the TNR, which reflects the classifier’s ability to detect the negative class, or, in our case, not churning customers:

TNR = 1/10 =0,1

Only a single observation from the negative class has been correctly classified as such (i.e. 1 customer who did not churn has been classified as not churning, and the remaining not churning customers has been classified as churning!).

**Conclusion**: ACC cannot be used to determine the extent to which the classes can be identified by the classifier.

In the subsequent parts of the tutorial we will introduce graphical indicators of classification model quality. We will start with the Confusion Matrix, which is a simple way of representing classification results.

Interested in similar content? Sign up for Newsletter

You can follow us at @Algolytics

© 2019 Data Science Central ® Powered by

Badges | Report an Issue | Privacy Policy | Terms of Service

**Most Popular Content on DSC**

To not miss this type of content in the future, subscribe to our newsletter.

**Technical**

- Free Books and Resources for DSC Members
- Learn Machine Learning Coding Basics in a weekend
- New Machine Learning Cheat Sheet | Old one
- Advanced Machine Learning with Basic Excel
- 12 Algorithms Every Data Scientist Should Know
- Hitchhiker's Guide to Data Science, Machine Learning, R, Python
- Visualizations: Comparing Tableau, SPSS, R, Excel, Matlab, JS, Pyth...
- How to Automatically Determine the Number of Clusters in your Data
- New Perspectives on Statistical Distributions and Deep Learning
- Fascinating New Results in the Theory of Randomness
- Long-range Correlations in Time Series: Modeling, Testing, Case Study
- Fast Combinatorial Feature Selection with New Definition of Predict...
- 10 types of regressions. Which one to use?
- 40 Techniques Used by Data Scientists
- 15 Deep Learning Tutorials
- R: a survival guide to data science with R

**Non Technical**

- Advanced Analytic Platforms - Incumbents Fall - Challengers Rise
- Difference between ML, Data Science, AI, Deep Learning, and Statistics
- How to Become a Data Scientist - On your own
- 16 analytic disciplines compared to data science
- Six categories of Data Scientists
- 21 data science systems used by Amazon to operate its business
- 24 Uses of Statistical Modeling
- 33 unusual problems that can be solved with data science
- 22 Differences Between Junior and Senior Data Scientists
- Why You Should be a Data Science Generalist - and How to Become One
- Becoming a Billionaire Data Scientist vs Struggling to Get a $100k Job
- Why do people with no experience want to become data scientists?

**Articles from top bloggers**

- Kirk Borne | Stephanie Glen | Vincent Granville
- Ajit Jaokar | Ronald van Loon | Bernard Marr
- Steve Miller | Bill Schmarzo | Bill Vorhies

**Other popular resources**

- Comprehensive Repository of Data Science and ML Resources
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- 100 Data Science Interview Questions and Answers
- Cheat Sheets | Curated Articles | Search | Jobs | Courses
- Post a Blog | Forum Questions | Books | Salaries | News

**Archives**: 2008-2014 | 2015-2016 | 2017-2019 | Book 1 | Book 2 | More

**Most popular articles**

- Free Book and Resources for DSC Members
- New Perspectives on Statistical Distributions and Deep Learning
- Time series, Growth Modeling and Data Science Wizardy
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- Comprehensive Repository of Data Science and ML Resources
- Advanced Machine Learning with Basic Excel
- Difference between ML, Data Science, AI, Deep Learning, and Statistics
- Selected Business Analytics, Data Science and ML articles
- How to Automatically Determine the Number of Clusters in your Data
- Fascinating New Results in the Theory of Randomness
- Hire a Data Scientist | Search DSC | Find a Job
- Post a Blog | Forum Questions

## You need to be a member of Data Science Central to add comments!

Join Data Science Central