Optimal Distributed Model Averaging for Multivariate Additive Model

In the era of massive data, the study of distributed data is a significant topic. Model averaging can be effectively applied to distributed data by combining information from all machines. For linear models, the model averaging approach has been developed in the context of distributed data. However, further investigation is needed for more complex models. In this paper, the authors propose a distributed optimal model averaging approach based on multivariate additive models, which approximates unknown functions using B-splines allowing each machine to have a different smoothing degree. To utilize the information from the covariance matrix of dependent errors in multivariate multiple regressions, the authors use the Mahalanobis distance to construct a Mallows-type weight choice criterion. The criterion can be computed by transmitting information between the local machines and the center machine in two steps. The authors demonstrate the asymptotic optimality of the proposed model averaging estimator when the covariates are subject to uncertainty, and obtain the convergence rate of the weight vector to the theoretically optimal weights. The results remain novel even for additive models with a single response variable. The numerical examples show that the proposed method yields good performance.