TransBLS: transformer combined with broad learning system for facial beauty prediction

Facial beauty prediction (FBP) is a frontier topic in the fields of machine learning and computer vision, focusing on how to enable computers to judge facial beauty like humans. The existing FBP methods are mainly based on deep neural networks (DNNs). However, DNNs lack global characteristics and only build local dependencies, so FBP still suffers from insufficient supervision information, low accuracy and overfitting. A transformer is a self-attention-based architecture that possesses better global characteristics than DNNs and can build long-term dependencies. Transformers have been widely used to solve some computer vision problems in recent years and have produced better results. In this paper, we propose an adaptive transformer with global and local multihead self-attention for FBP, called GLAFormer. However, GLAFormer does not converge and is prone to overfitting when the training samples are insufficient. The broad learning system (BLS) can accelerate the model convergence process and reduce overfitting. Therefore, we further combine GLAFormer with the BLS to form TransBLS, in which a GLAFormer block is designed as a feature extractor, the features extracted by it are transferred to the BLS for further refining and fitting, and the results are output. Experimental results indicate that TransBLS achieves state-of-the-art FBP performance on several datasets with different scales, better solving the low accuracy and overfitting problems encountered in FBP. It can also be widely applied in pattern recognition and object detection tasks.