Ensemble learning of linear perceptrons: On-line learning theory

We analyze ensemble learning including the noisy case where teacher or student noise is present. Linear perceptrons are used as teacher and student. First, we analyze the homogeneous correlation of initial weight vectors. The generalization error consists of two parts: the first term depends on the number of perceptrons K and is proportional to 1/K, the second does not depend on K in the first case. In the inhomogeneous correlation of initial weight vectors case, the weighted average could be optimized to minimize the generalization error. We found that the optimal weights do not depend on time without student noise, while the optimal weights depend on time and become 1/K with student noise.