In the last part of the tutorial we introduced the basic qualitative model quality indicators. Let us recall them now:
We will now discuss derived variants of these indicators.
TPR = TP / (TP + FN)
TNR = TN / (TN + FP)
FPR = FP / (FP + TN)
FNR = FN / (FN + TP)
SE = TP / (TP + FN)
SP = TN / (TN + FP)
Note that the following equalities hold:
SE = TPR 1 – SE = FNR
SP = TNR 1-SP = FPR
ACC = (TP + TN) / (TP + TN + FP + FN)
There are numerous measures of classifier quality and it is necessary to select the one which is appropriate for the situation at hand.
Example
During churn analysis we have two classes in the data: churn and no churn. There are 100 observations in the data and
For a certain classifier we obtain:
TP = 85 (85 churning customers have been classified as)
TN = 1 (1 not churning customer has been classified as such)
FP = 9 (9 not churning customers have been classified as churning)
FN = 5 (5 churning customers have been classified as not churning)
The sum of the above indicators equals the number of observations (customers):
TP + TN + FP + FN = 100
We now calculate the total accuracy of the classifier:
ACC = (85+1)/100 = 0,86
86 cases out of 100 has been correctly classified, which is not a bad result at all.
However, let us take a look at another indicator, the TNR, which reflects the classifier’s ability to detect the negative class, or, in our case, not churning customers:
TNR = 1/10 =0,1
Only a single observation from the negative class has been correctly classified as such (i.e. 1 customer who did not churn has been classified as not churning, and the remaining not churning customers has been classified as churning!).
Conclusion: ACC cannot be used to determine the extent to which the classes can be identified by the classifier.
In the subsequent parts of the tutorial we will introduce graphical indicators of classification model quality. We will start with the Confusion Matrix, which is a simple way of representing classification results.
Interested in similar content? Sign up for Newsletter
You can follow us at @Algolytics
© 2021 TechTarget, Inc.
Powered by
Badges | Report an Issue | Privacy Policy | Terms of Service
Most Popular Content on DSC
To not miss this type of content in the future, subscribe to our newsletter.
Other popular resources
Archives: 2008-2014 | 2015-2016 | 2017-2019 | Book 1 | Book 2 | More
Most popular articles
You need to be a member of Data Science Central to add comments!
Join Data Science Central