FedEntropy: Information-entropy-aided training optimization of semi-supervised federated learning

Emerging federated learning (FL) is able to train a global machine learning (ML) model by using decentralized data from various clients, without exposing the privacy data of clients. Traditional FL assumes that the training data are labeled, but in reality the data captured by the clients are usually unlabeled. Nowadays, the manual data labeling, a common method, is very expensive in practical operation. To solve the above problems, in this paper, we propose a semi-supervised federated learning scheme (FedEntropy) to improve the model performance in the case that unlabeled data dominates the datasets. Specifically, our proposed FedEntropy firstly utilizes information entropy to jointly compute the loss of labeled and unlabeled data. Subsequently, assisted with inverse-trigonometric-based adaptive proportional adjustment algorithm, FedEntropy is able to dynamically set the ratio between loss of labeled and unmarked data. In particular, we prove the effectiveness of the information entropy function on unlabeled data training and reducing the probability distribution gap of datasets. Extensive experiments results demonstrate that, compared with state of art methods, our FedEntropy not only achieves accuracy improvement of up to 6.42% on two common datasets, but also simultaneously reduces the approximately half of the computation overheads in semi-supervised FL training.