An Open Source Workbench for Prototyping Multimodal Interactions Based on Off-The-Shelf Heterogeneous Components

Lawson, Jean-Yves Lionel [UCL] Al-Akkad, Ahmad-Amr [Fraunhofer FIT] Vanderdonckt, Jean [UCL] Macq, Benoît [UCL]

In this paper we present an extensible software workbench for supporting the effective and dynamic prototyping of multimodal interactive systems. We hypothesize the construction of such applications to be based on the assembly of several components, namely various and sometimes interchangeable modalities at the input, fusion-fission components, and also several modalities at the output. Successful realization of advanced interactions can benefit from early prototyping and the iterative implementation of design requires the easy integration, combination, replacement, or upgrade of components. We have designed and implemented a thin integration platform able to manage these key elements, and thus provide the research community a tool to bridge the gap of the current support for multimodal applications implementation. The platform is included within a workbench offering visual editors, non-intrusive tools, components and techniques to assemble various modalities provided in different implementation technologies, while keeping a high level of performance of the integrated system