Multi-level knowledge representation in neural networks with adaptive structure

Principles of learning and knowledge representation in complex systems are studied here. The description of the behavior of complex systems requires models with a large number of parameters. Very often, however, the actual modeling, control, etc. tasks can be completed by considering only a narrow subspace of the high-dimensional hyperspace of all system variables. This subspace, in which the actual analysis is carried out, might change with time if the external environment of the analyzed system varies. In the paper artificial neural networks are introduced which are capable to develop both deep and shallow knowledge representations in response to changes in the environment. The networks utilize structural learning to train feed-forward neural nets with multi-layer architecture. The applied training algorithms result in a saturated skeleton network structure which is not frozen but exhibits small fluctuations around its equilibrium. Small fluctuations of the weights can grow into a structural evolution in the neural net if properties of the input clusters change. This feature is especially advantageous in on-line learning when a rigid neural network structure could lead to mis-judgments among dynamically changing conditions. Structural adaptation features are illustrated using Fisher's IRIS data and the analysis of non-stationary time series.