On computational limitations of self-organization

The idea of self-organization has attracted a lot of interest from scientists in different disciplines, including Computer Science, Neuroscience, Cognitive Science, and Philosophy. The complex dynamics occurring in Neural Network-style self-organizing systems pose an extraordinary difficulty for an analysis of the potential and limitations of such systems, However, this paper provides rigorous bounds on the complexity of structures which may be emerging in self-organizing systems. It is shown how the notion of algorithmic information theory can elegantly be used to analyze the complex dynamics of parallel and distributed systems. First, results using this method are presented. Conditions are identified under which the complexity of a meaningful structure that can possibly emerge, is severely limited by the complexity of the system before it is exposed to training data! On the other hand, conditions are identified under which the acquisition of complex functions is possible. The results provide insight into the factors which may enable successful learning from unclassified data. They also indicate that the conditions under which a self-organizing system can be successfully employed to construct truly complex and useful classification functions require careful examination through further research.