Subscribe to DSC Newsletter

Removing Confusion From Confusion Matrix — Hawaii False Missile Alert

“Missile approaching, Every one run for cover”. People sending final goodbye messages to their loved ones. Family members huddling and praying together. Mother clinging to children, while the father hurriedly makes for a safe getaway for the family to the basement. Imagine the state of confusion and chaos every Hawaiian would have faced on Jan 13, 2018 when they received an approaching ballistic missile alert.

Talk about trouble in paradise !!

Well given the current twitter threats by ‘Rocket Man’ as President Trump calls the North Korean dictator, many Hawaiian’s had all the logical reasons to believe the alert. Thankfully ‘Rocket Man’ had not pressed the tiny nuclear button probably because Trump had a bigger button !!

It all thankfully turned out to be a ‘False Alarm’. Apart from minor car accidents ,unfortunate heart attack suffered by a person and not to mention the drastic drop in pornhub views and sudden surge in pornhub views (“Stronger than ever before, EVER” in Trump speak), there was not much of a damage.

What if the missile alert was real ? The mere thought of it is scary enough.

During the world war 2, the British and allies were going through the same confusion and fear everyday. Something needed to be done.

British developed the ROC curve as part of the chain home project. The goal was to discern which was an incoming enemy plane and which was a seagull or Geese.

The Receiver Operating Curve (ROC) is a graphical plot that depicts the diagnostic ability of a binary classification system with varying threshold. It is a plot of True Positive Rate against False Positive Rate.

I know the terms False Positive Rate and True Positive rate might be confusing right now for some people, but the reason I bought the ROC into picture is because it will help set the narrative later on in the article for some interesting debunking about Missile Alert UI design.

What Causes the Confusion in Confusion Matrix ?

In my opinion the confusion gets created the moment we place words like predicted class on the left and actual class on top as illustrated below.

Our human brain gets into ‘match the following’ mode and gets confused due to the sheer cognitive burden.

How To Remove The Confusion ?

I must confess that I too used to get confused between False Positive and False Negative while building and interpreting various classification models, until i developed my own technique to remember it for life time. It works for me and perhaps it might for you too.

It is paramount to get the TP, TN, FP, FN right as they prove crucial in calculating TPR (a.k.a sensitivity), TNR (a.k.a Specificity) and many more. I will probably write an article with real examples on how to calculate these ratios without getting confused.

But first things first, steps to remove the confusion.

Step 1 : Keep the word “Predicted” in mind always

Step 2 : Read from right to left (Arabs and Persians might find this easier)

Step 3: If the word is FP, you should read it as Predicted Positive but False

Similarly, say the word is FN, you would read it as Predicted Negative but False

I don’t want to delve into TP (True Positives) and TN (True Negatives) as they are self explanatory.

Was the Hawaii Missile Alert System Designed Wrongly ?

Many articles have been written on how bad the UI was and how easily anyone could have made the mistake. The options were in the form of a drop down

  • “Test Missile Alert”
  • “Missile Alert”

Was the system badly designed ?

My answer with a Data Scientist Hat on, would be NO.

Here’s why :

The UI was designed to be a battle of True Positives (TP) vs False Positives (FP)

Remember the ROC Curve I had talked about earlier ? In a life or death situations, you would want to be alert all the time. In a battle between True Positive and False Positive, the winner is always YOU. Think about it. What is the worst that can happen.

Only two possibilities:

  1. True Positive : A Real missile is approaching and you take shelter
  2. False Positive : No missile approaching, yet the system says it is, and you take shelter

Either case you take shelter. You are safe!!

What if the battle was between False Negative (FN) vs True Negative (TN)

Well this is one battle were you want to root for True Negative always. why ?

Again two possibilities:

  1. False Negative : A Real Missile is approaching, yet system says not a speck in the sky. You don’t take shelter and………..(eternal silence)
  2. True Negative : No missile is approaching and No alerts. You are safe!!

So whosoever designed the UI did a good job. It will save more lives than otherwise.

Bottom line : Life or Death situation alert systems should always be designed with True Positives (TP)and False Positives (FP) options alone.


Hope you liked my article. You can also comment below your opinions about the article.

You can reach out to me on

Linkedin

Twitter

Views: 1386

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Curwen Arthurs on January 31, 2018 at 5:43am

Yes Thanks. It's clear now. The True and False does not refer to the condition being true or false but rather it refers to your prediction(assumption) being true or false.

Looking forward to seeing your next article on calculating the ratios

Comment by Venkat Raman on January 30, 2018 at 6:52pm

Thanks Curwen for your comments. Glad to know you liked my article. I have covered TN and FN correctly. Let me try to clear your confusion.

TP :The positive event here is Missile Really approaching. So TP is the missile really approaching and the system detects it.

FP : No missile is approaching but the system says a missile is approaching. (What happened in Hawaii recently)

True Negative : No missile is approaching and the system also says no missile approaching. 

False Negative : System says no missile is approaching but in reality a missile is approaching.

In TN and FN the negative case is "Missile not approaching". Now refer to step 3 and step 4 in the article, it should make it clear to you. 

Hope I have clarified your confusion ?

Comment by Curwen Arthurs on January 30, 2018 at 1:51pm

Thanks for the contribution Venkat. It was a very interesting read. 

I understand the concept of True Positive and False Positives; however its the True Negative and False Negative that are confusing. I had thought that True Negative meant that the condition is true(present) but the test did not detect it and that False Negative meant that the condition is false(absent) and the test did not detect it.

So the last 2 possibilities you mentioned would have been in reverse order.

True Negative = A real missile is approach but you don't take shelter

False Negative = No missile is approaching and No alerts. You are safe!! 

Follow Us

Videos

  • Add Videos
  • View All

Resources

© 2018   Data Science Central™   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service