Online Learning of Perceptron from Noisy Data: A Case in which Both Student and Teacher Suffer from External Noise

We analyze the online learning of a Perceptron (student) from signals produced by a single Perceptron (teacher) in which both the student and the teacher suffer from external noise. We adopt three typical learning rules and treat the input and output noises. In order to improve learning when it fails in the sense that the student vector does not converge to the teacher vector, we use a method based on the optimal learning rate. Furthermore, in order to control learning, we propose a concrete method for the Perceptron rule in the output noise model. Finally, we analyze time domain ensemble online learning. The theoretical results agree quite well with the numerical simulation results.