Information theoretic competitive learning in self-adaptive multi-layered networks

In this paper, we propose self-adaptive multi-layered networks in which information in each processing layer is always maximized. Using these multi-layered networks, we can solve complex problems and discover salient features that single-layered networks fail to extract. In addition, this successive information maximization enables networks gradually to extract important features. We applied the new method to the Iris data problem, the vertical-horizontal lines detection problem, a phonological data analysis problem and a medical data problem. Experimental results confirmed that information can repeatedly be maximized in multi-layered networks and that the networks can extract features that cannot be detected by single-layered networks. In addition, features extracted in successive layers are cumulatively combined to detect more macroscopic features.