Adaptive Active Learning with Ensemble of Learners and Multiclass Problems

Active Learning (AL) is an emerging field of machine learning focusing on creating a closed loop of learner (statistical model) and oracle (expert able to label examples) in order to exploit the vast amounts of accessible unlabeled datasets in the most effective way from the classification point of view. This paper analyzes the problem of multiclass active learning methods and proposes to approach it in a new way through substitution of the original concept of predefined utility function with an ensemble of learners. As opposed to known ensemble methods in AL, where learners vote for a particular example, we use them as a black box mechanisms for which we try to model the current competence value using adaptive training scheme. We show that modeling this problem as a multi-armed bandit problem and applying even very basic strategies bring significant improvement to the AL process.