A Self-Learning Sensor Fusion System for Object Classification

We propose a learning system for object classification which fuses information from a camera, a radar and a localization unit. The system is illustrated in application to categorization of objects on a highway. The system learns not only prior to its deployment in a supervised mode but also on-board a vehicle during its operation in a self-learning mode. The radar guides a selection of candidate images provided by the camera for subsequent analysis by our learning method. The Multilayer In-place Learning Network (MILN) is used to distinguish between representations of different objects. Radar information gets coupled with navigational information for accurate localization of objects during self-learning. One of the MILN layers helps to resolve labeling conflicts when localization is not sufficient. A Multi-Resolution MILN which uses higher-resolution levels to reinforce training of lower-resolution levels is also proposed for improved performance when dealing with a wide range of distances to objects.