Category-aware self-training for extremely weakly supervised text classification

The popular forms of text classification under extremely weakly supervised employ a two-phase pipeline. The first is the generation of pseudo-labeled data, followed by a self-training phase. In the pseudo-labeling phase, it primarily obtains initial pseudo-labels through pre-trained models or existing unsupervised methods. After this phase, the pseudo-labeled training set still contains some noise. Therefore, the self-training process plays a key role in influencing the final performance. However, during the self-training process, it was observed that many existing approaches struggle to capture key information in documents without any guidance, resulting in the acquisition of erroneous knowledge. Therefore, this paper proposes a category-aware self-training process that can guide the model to purposefully acquire key information from documents. Specifically, rather than merely utilizing text-based word vector features as in previous models, it delves into the category perspective, directing the model to learn both category-related word vector features and statistical attributes. This enables the model to refit based on these coarse-grained features and label information, providing more opportunities for the model to locate key information in documents. Furthermore, by applying Gaussian processing to the intermediate features, the model's robustness is also enhanced. Finally, we conducted extensive testing on multiple publicly available datasets. Experimental results demonstrate that our proposed method outperforms existing approaches in extremely weakly supervised text classification.