Quite often in Data Science we deal with use cases where the penalty for False Negative can be huge. Examples can be missing to predict having a disease, denying loan based on false prediction, not acting on a cyber threat etc. 

So the question is what kind of techniques exist to minimize or eliminate false negatives?

Views: 926

Reply to This

Replies to This Discussion

How do those links relate to the topic?

This is not just about minimizing false negatives, but rather dealing with imbalanced classes. Some techniques to deal with this are:

- use the right evalualtion metrics - basically do not just use accuracy as it is rather deceiving for the imbalanced class problem; use AUC or f-score instead.

- down-sample or up-sample corresponding classes to have a more balanced set.

- there is a scale_pos_weight parameter in most of the decision tree based classifiers to adjust the weights of classes; play with this to get higher accuracy on your desired class.

Thanks Adil

Remember that a False Positive (FP) or False Negative (FN) is essentially caused by the output variable (the classification) overlapping in a low-dimension projection of the feature space of input variables. Thus, one way to reduce FP and FN is to introduce higher dimensions (more features, more tests, greater perspectives). Just like a doctor orders more tests (more test results = more data features to use in the diagnosis) when there is confusion in a diagnosis, a data scientist should "order more data" = more features, more dimensions. Look here: https://datamakespossible.westerndigital.com/five-ds-big-data-variety/

Thanks Kirk


© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service