.

Data Science Techniques to eliminate False Negatives

Quite often in Data Science we deal with use cases where the penalty for False Negative can be huge. Examples can be missing to predict having a disease, denying loan based on false prediction, not acting on a cyber threat etc. 

So the question is what kind of techniques exist to minimize or eliminate false negatives?

Views: 857

▶ Reply to This

Replies to This Discussion

Permalink Reply by Marissa Stice on October 23, 2020 at 10:21pm
Permalink Reply by Arunansu Pattanayak on October 26, 2020 at 6:46am

How do those links relate to the topic?

Permalink Reply by Adil Gursel Karacor on October 24, 2020 at 1:02am

This is not just about minimizing false negatives, but rather dealing with imbalanced classes. Some techniques to deal with this are:

- use the right evalualtion metrics - basically do not just use accuracy as it is rather deceiving for the imbalanced class problem; use AUC or f-score instead.

- down-sample or up-sample corresponding classes to have a more balanced set.

- there is a scale_pos_weight parameter in most of the decision tree based classifiers to adjust the weights of classes; play with this to get higher accuracy on your desired class.

Permalink Reply by Arunansu Pattanayak on October 26, 2020 at 6:47am

Thanks Adil

Permalink Reply by Kirk Borne on October 26, 2020 at 2:35pm

Remember that a False Positive (FP) or False Negative (FN) is essentially caused by the output variable (the classification) overlapping in a low-dimension projection of the feature space of input variables. Thus, one way to reduce FP and FN is to introduce higher dimensions (more features, more tests, greater perspectives). Just like a doctor orders more tests (more test results = more data features to use in the diagnosis) when there is confusion in a diagnosis, a data scientist should "order more data" = more features, more dimensions. Look here: https://datamakespossible.westerndigital.com/five-ds-big-data-variety/

Permalink Reply by Arunansu Pattanayak on October 28, 2020 at 11:13am

Thanks Kirk

RSS

Related TechTarget Content

Posted 13 July 2021

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service