Classifiers with low decision-making error using linear combination of functions

Usually in classification, the definition of the "error rate" does not differentiate an element misclassified from an element not classified. However, in some applications as medical diagnosis, it is better not to classify rather than to make a mistake. In such a case, a human can classify the element non classified by the learning system, eventually after further investigations (e.g. in the medical case, a deeper evaluation of patient history). In this paper, we will define the decision-making error as the conditional probability that an element is misclassified knowing it is classified. We propose an algorithm, based on convex linear combination of classifiers, in order to improve the decision-making error without increasing too much the not classified rate. We derive theoretical statistical results, on confidence bounds for the generalization performance of linear combination of functions, to give confidence bounds for our algorithm.