HandTill2001
A simple generalisation of the area under the ROC curve for multiple class classification problems The area under the ROC curve, or the equivalent Gini index, is a widely used measure of performance of supervised classification rules. It has the attractive property that it side-steps the need to specify the costs of the different kinds of misclassification. However, the simple form is only applicable to the case of two classes. We extend the definition to the case of more than two classes by averaging pairwise comparisons. This measure reduces to the standard form in the two class case. We compare its properties with the standard measure of proportion correct and an alternative definition of proportion correct based on pairwise comparison of classes for a simple artificial case and illustrate its application on eight data sets. On the data sets we examined, the measures produced similar, but not identical results, reflecting the different aspects of performance that they were measuring. Like the area under the ROC curve, the measure we propose is useful in those many situations where it is impossible to give costs for the different kinds of misclassification.
Keywords for this software
References in zbMATH (referenced in 67 articles , 1 standard article )
Showing results 1 to 20 of 67.
Sorted by year (- Conde, David; Fernández, Miguel A.; Rueda, Cristina; Salvador, Bonifacio: Isotonic boosting classification rules (2021)
- Fu, Chao; Liu, Weiyong; Chang, Wenjun: Data-driven multiple criteria decision making for diagnosis of thyroid cancer (2020)
- Halbersberg, Dan; Wienreb, Maydan; Lerner, Boaz: Joint maximization of accuracy and information for learning the structure of a Bayesian network classifier (2020)
- Ippel, L.; Kaptein, M. C.; Vermunt, J. K.: Online estimation of individual-level effects using streaming shrinkage factors (2019)
- Plaia, Antonella; Sciandra, Mariangela: Weighted distance-based trees for ranking data (2019)
- Schechtman, Edna; Schechtman, Gideon: The relationship between Gini terminology and the ROC curve (2019)
- Zhang, Yongshan; Wu, Jia; Cai, Zhihua; Du, Bo; Yu, Philip S.: An unsupervised parameter learning model for RVFL neural network (2019)
- Huang, Hsin-Hsiung; Hao, Shuai; Alarcon, Saul; Yang, Jie: Comparisons of classification methods for viral genomes and protein families using alignment-free vectorization (2018)
- Ma, Hua; Bandos, Andriy I.; Gur, David: Informativeness of diagnostic marker values and the impact of data grouping (2018)
- Meng, Yinfeng; Liang, Jiye; Cao, Fuyuan; He, Yijun: A new distance with derivative information for functional (k)-means clustering algorithm (2018)
- Probst, Philipp; Boulesteix, Anne-Laure: To tune or not to tune the number of trees in random forest (2018)
- Zhang, Wenyu; Zhang, Zhenjiang; Chao, Han-Chieh; Tseng, Fan-Hsun: Kernel mixture model for probability density estimation in Bayesian classifiers (2018)
- Ting, Kai Ming; Washio, Takashi; Wells, Jonathan R.; Aryal, Sunil: Defying the gravity of learning curve: a characteristic of nearest neighbour anomaly detectors (2017)
- Stein, Roger M.: Evaluating discrete choice prediction models when the evaluation data is corrupted: analytic results and bias corrections for the area under the ROC (2016)
- Wang, Shijun; Li, Diana; Petrick, Nicholas; Sahiner, Berkman; Linguraru, Marius George; Summers, Ronald M.: Optimizing area under the ROC curve using semi-supervised learning (2015)
- Daqi, Gao; Jun, Ding; Changming, Zhu: Integrated Fisher linear discriminants: an empirical study (2014)
- Dong, Enming; Li, Jianping; Xie, Zheng: Link prediction via convex nonnegative matrix factorization on multiscale blocks (2014)
- Feng, Guang; Zhang, Jia-Dong; Shaoyi Liao, Stephen: A novel method for combining Bayesian networks, theoretical analysis, and its applications (2014)
- He, Yu-Lin; Wang, Ran; Kwong, Sam; Wang, Xi-Zhao: Bayesian classifiers based on probability density estimation and their applications to simultaneous fault diagnosis (2014)
- Katz, Gilad; Shabtai, Asaf; Rokach, Lior; Ofek, Nir: ConfDTree: a statistical method for improving decision trees (2014) ioport