Rule extraction from support vector machines A review

Over the last decade support vector machine classifiers (SVMs) have demonstrated superior generalization performance to many other classification techniques in a variety of application areas However SVMs have an inability to provide an explanation or comprehensible justification for the solutions they reach It has been shown that the black-box nature of techniques like artificial neural networks (ANNs) is one of the main obstacles impeding their practical application Therefore techniques for rule extraction from ANNs and recently from SVMs were introduced to ameliorate this problem and aid in the explanation of their classification decisions In this paper we conduct a formal review of the area of rule extraction from SVMs The review provides a historical perspective for this area of research and conceptually groups and analyzes the various techniques In particular we propose two alternative groupings the first is based on the SVM (model) components utilized for rule extraction while the second is based on the rule extraction approach The aim is to provide a better understanding of the topic in addition to summarizing the main features of individual algorithms The analysis is then followed by a comparative evaluation of the algorithms salient features and relative performance as measured by a number of metrics It is concluded that there is no one algorithm that can be favored in general However methods that are kernel independent produce the most comprehensible rule set and have the highest fidelity to the SVM should be preferred In addition a specific method can be preferred if the context of the requirements of a specific application so that appropriate tradeoffs may be made The paper concludes by highlighting potential research directions such as the need for rule extraction methods in the case of SVM incremental and active learning and other application domains where special types of SVMs are utilized (C) 2010 Elsevier B V All rights reserved