Multi-modal interaction in AAL systems

Objective Ambient Assisted Living is regarded as a promising tool to support independent life at home. CARDEA [1] is an environmental monitoring and control system suitable for implementing AAL services, aimed at low cost and fostering device interoperability. The adoption of standardized and massively diffused communication protocols, together with the implementation of hardware abstraction strategies, enhances CARDEA versatility and makes it open to "plugin" modules for implementing additional features. In this abstract, CARDEA extensions are presented, aimed at making home control functionalities accessible to people with severe motion and communication impairment. In particular, voice-and brain-control modules were added to CARDEA. The voice extension is based on open-source speech recognition tools, allowing the user for introducing his own acoustic and language models. A set of models suitable for the locale has been developed, and the speech recognizer has been connected to the CARDEA control module. The adoption of an open-source module allows for more flexibility in the system personalization, reducing the need for user training. The brain-control module is based on an inexpensive, single-channel EEG device, originally conceived for entertainment purposes. Different control techniques were investigated: first by looking at amplitude of "emotion" signal worked out by the BCI system, which can be controlled by the user, after a short training, by means of a visual feedback. A scan interface can be used for selecting one choice among different options. Although working, the system suffers from the relatively long time needed for attaining the required amplitude threshold, which need to be high enough to enable reliable command recognition. Hence, a different approach has been investigated, based on the SSVEP (Steady-State Visually Evoked Potential) analysis. When the user's retina is exposed to a low-frequency stimulus, brainwaves exhibit the onset of correlated frequency components. The user is shown a set of modulated light sources, each light corresponding to a given action. Then he simply look at the light corresponding to the required action for a few seconds: by means of spectral decomposition of the brainwave signals, the system infers the chosen option and triggers the corresponding action. With respect to the amplitude-based approach, such a frequency-domain approach is significantly faster and requires almost no training to the user. Moreover, being not based on scan sequences, it is more suitable for the implementation of complex decision trees. On the other hand, it obviously requires the user retains visual ability and needs control of the extraocular muscles. All of the above approaches were tested in a lab environment with a set of volunteers. Voice and brain interfaces were connected to CARDEA, allowing for controlling a demo room. So doing, users were able to trigger simple functions (such as switching on or off an appliance) in many different ways. Lab test shows fairly satisfactory performance and functionality for the proposed techniques, making them promising solutions for effective, low-cost implementation of multi-modal interaction in AAL environments.