Cost-Sensitive Classifier Evaluation Using Cost Curves

Abstract: The evaluation of classifier performance in a cost-sensitive setting is straightforward if the operating conditions (misclassification costs and class distributions) are fixed and known. When this is not the case, evaluation requires a method of visualizing classifier performance across the full range of possible operating conditions. This talk outlines the most important requirements for cost-sensitive classifier evaluation, and introduces a technique for classifier performance visualization – the cost curve – that meets all these requirements. This talk should be of interest to anyone who works in areas that use classifiers, for example, machine learning, pattern recognition, biometrics, and diagnosis.

Bio: Professor Robert Holte of the Computing Science Department at the University of Alberta is a former editor-in-chief of the journal “Machine Learning” and co-founder and former director of the Alberta Innovates Center for Machine Learning (AICML, now known as Amii). His current research is on single-agent heuristic search, with seminal contributions on bidirectional search, methods for predicting the run-time of a search algorithm, and the use of machine learning to create heuristics. Professor Holte was elected a Fellow of the AAAI in 2011.