A Transformational Approach for Multimodal Web User Interfacesbased on UsiXML

Stanciulescu, Adrian Limbourg, Quentin Vanderdonckt, Jean Michotte, Benjamin Montero Simarro, Franscisco

A transformational approach for developing multimodal web user interfaces is presented that progressively moves from a task model and a domain model to a final user interface. This approach consists of three steps: deriving one or many abstract user inter-faces from a task model and a domain model, deriving one or many concrete user interfaces from each abstract one, and produc-ing the code of the corresponding final user interfaces. To ensure these steps, transformations are encoded as graph transformations performed on the involved models expressed in their graph equivalent. For each step, a graph grammar gathers relevant graph transformations for accomplishing the sub-steps. The final user interface is multimodal as it involves graphical (keyboard, mouse) and vocal interaction. The approach outlined in the paper is illus-trated throughout a running example for a graphical interface, a vocal interface, and two multimodal interfaces with graphical and vocal predominances, respectively.