Active Learning with few-shot learning for crisis management

Practical applications of machine learning require fast training of models with few annotations on specific domains. Among them, crisis management is of prime importance to support decisions by extracting useful insights from massive datasets. In particular, social media analysis is an intense topic in this domain. In this paper, we investigate whether this problem can be solved by combining an active learning strategy and the few-shot learning paradigm enabled by recent language models. We find that although Few-Shot learning based on K nearest neighbors and greedy Core-set obtain superior performances at a very low data regime, an active learning strategy based on fine-tuning a Distillbert model performs best as more points are observed, still staying in the regime of less than a hundred of points per class. However, the computation burden of the few-shot based method is typically much lower. These findings are obtained through experiments on Twitter data with the CrisisMMD public dataset.