Evaluation of the sensorimotor learning in gesture-sound interactive systems – LEGOS

The considerable expansion of gesture interfaces in human-machine interfaces, from mobile computing systems to video games, constantly brings novel technological systems and usages where the user actions are central. In fact, gesture control in interactive sound systems has considerably grown over the last ten years, both conceptually and technologically.
The central idea of the LEGOS project is to fertilize expertises in gesture controlled sound technologies with neurosciences, especially regarding sensori-motor learning. We believe that these aspects are not sufficiently taken into account in the developments of interactive sound systems. A better understanding of the sensori-motor learning mechanisms of the gesture / sound coupling is necessary to provide efficient methodologies for their evaluation and optimization.
Such advances would significantly expand the usability of today's gesture-based interactive sound systems, often developed empirically.

The objective of the LEGOS project is to study systematically the coupling quality in gesture-sound systems using gestural interfaces. For this, we will evaluate the sensory-motor learning, and particularly its evolution over time, in various interactive devices. The aims are therefore to develop, evaluate and compare interactive systems. The mid-term goal is to offer new paradigms for digital media gestural interfaces (including video games and sound design), as well as prototypes for medical applications such as rehabilitation.

The project will make use extensively of an experimental approach, considering these three perspectives
1) "Sound control": The first point corresponds to a case of sensorimotor learning where the goal is to produce a given sound through the manipulation of a gestural interface, as in the case of digital musical instruments. The sensorimotor learning is assessed in terms of the quality of the sound production.
2) "Learning to gesture with audio feedback" This second point corresponds to the sensorimotor learning where the goal is to make a gesture guided by an audio feedback. The sensorimotor learning in this case is assessed in terms of the gesture repeatability
3) "Interactive Sound Design": This third point corresponds to sensorimotor learning in the case of tangible interfaces, where the goal is the proper handling of an object. The sensorimotor learning in this case is assessed through the quality of the objet manipulation.
The joined analysis of the experimental results should provide us with a comprehensive perspective, leading to novel methodologies for the gestural interfaces design.

LEGOS is thus an interdisciplinary project dedicated to the study and evaluation of learning sensory-motor in interactive audio systems. This research topic is primarily linked to important themes in the fields of ICT (Information and Communication Technologies), human-machine interfaces, virtual reality and augmented reality, sound and music computing. IRCAM (Institute for Research and Coordination in Acoustics / Music , STMS UMR CNRS) is a leading institution on these topics at an international level, coordinating or participating in several national and European projects. IRCAM will be associated with the Neuromouv team (movement dynamics and sensory-motor learning) of Laboratory of Neurophysics and Physiology Paris 5 (UMR CNRS), which is recognized internationally in this neurosciences topic.