Binary classification is one of the most frequent studies in applied machine learning problems in various domains, from medicine to biology to meteorology to malware analysis. Many researchers use some performance metrics in their classification studies to report their success. However, the literature has shown a widespread confusion about the terminology and ignorance of the fundamental aspects behind metrics.

In our paper tittled "Binary Classification Performance Measures/Metrics: A Comprehensive Visualized Roadmap to Gain New Insights" clarifies the confusing terminology, suggests formal rules to distinguish between measures and metrics for the first time, and proposes a new comprehensive visualized roadmap in a leveled structure for 22 measures and 22 metrics for exploring binary classification performance.

Additionally, we introduced novel concepts such as canonical notation, duality, and complementation for measures/metrics, and suggested two new canonical base measures (TC and FC) simplifying equations.

It is expected that the study will guide other studies to have standardized approach to performance metrics for machine learning based solutions.

The roadmap can be considered as the "Periodic Table of Chemical Elements" in Machine Learning Binary Classification Performance Evaluation.

Figure: A glimpse of the binary classification performance measures/metrics

Canbek, G., Sagiroglu, S., Temizel, T. T., & Baykal, N. (2017). Binary Classification Performance Measures/Metrics: A Comprehensive Visualized Roadmap to Gain New Insights. In 2017 International Conference on Computer Science and Engineering (UBMK) (pp. 821–826). Antalya, Turkey: IEEE. http://doi.org/10.1109/UBMK.2017.8093539

IEEE Xplore: http://bit.ly/perfroadmap

Full-text at: http://phd.canbek.com/ArticlePerformanceMetricsRoadMap.pdf