Multimodal interfaces

The authors consider the scenario above as really important and propose a prototype for the virtual enterprises as the basis of a large scale virtual community. The prototype presents several advantages the multimodal interfaces guarantee, and combines the speech and ink formats to facilitate the creation of robust and efficient multimodal mobile E-Business systems. 

The customer s virtual assistant represents the system in the dialogue with the customer and implements the multimodal interface concept for both e-commerce and technical support sections. The role of the multimodal interface is to make the virtual market close to the real world the virtual assistant and the real vendor in the sales process are similar. The customer s virtual assistance also implements the call center capabilities it queries the business intelligence component that decides the type of problem the customer has, calls the technical support and establish the communication between the customer and one of the technical support specialists. 

The virtual store supports speech-based user interaction and 2D interactive graphics. The multimodal interface allows the customer to browse the catalogues, select a product and buy it using the own voice. The figure 6 illustrates the virtual assistant presenting a selected product to the customer. 

Keywords Multimodal interfaces, usability evaluation method, two mice, speech. 

Making interaction more natural is one of the promises of multimodal interfaces. Research on evaluating multimodal interfaces dates back to the mid 1980s [5] and since then it is seen as a promising approach to improve desktop interaction techniques. Multimodal interaction techniques are considered as a potential way to increase communication bandwidth between users and systems and to enhance users satisfaction and comfort by providing a more natural way of interacting with computer systems [5], [6]. 

To prove our hypothesis that two-handed interaction and speech is adopted quite easily and quickly by users, we performed a usability study. Usability evaluation studies seem to be a preferred strategy for the evaluation of multimodal interfaces because it allows the investigation of how users adopt and interact with multimodal technology providing valuable information about the general usability and the user experience. Several types of user testing have been conducted for multimodal interfaces in both usability laboratories and in the field revealing user preferences for interaction modalities based on factors such as acceptance in different social contexts, noisy and mobile environments [18]. An overview on new forms of usability evaluation methods for multimodal interfaces is presented in [4]. 

This system has been built within a research project funded by CNES (French National Study Center on Space). The project was meant at defining a formal description technique capable of describing multimodal interfaces and multimodal interaction techniques in a complete and unambiguous way. 

Oviatt, S.L., Coulston, R., Lunsford, R. When Do We Act Multimodally Cognitive Load and Multimodal Communication Patterns. In Sixth International Conference on Multimodal Interfaces (ICMI 2004), ACM Press, New York (2004)... 

The second application is to manipulate modalities in a visually-cluttered environment, i.e. an aircraft cockpit wherein interaction is safety-critical [6]. The chosen scenario is an aircraft descent scenario with interventions from the autopilot. The main objectives of this experiment are to investigate the pilot s attention drawn from rapidly changing cockpit (multimodal) interface by the autopilot and to discover the best perceptible modality for improving the overall safety for this environment. 

Oviatt, S.L. Multimodal interfaces. In Jacko, J., Sears, A. (eds.) The Human-Computer Interaction Handbook Fundamentals, Evolving Technologies and Emerging Applications, ch.l4, pp. 286-304. Lawrence Erlbaum Assoc, Mahwah (2003)... 

Bouchet, J., Nigay, L., Ganille, T. ICARE software components for rapidly developing multimodal interfaces. In Proceedings of the 6th international conference on Mnltimodal interfaces ICMI 2004, pp. 251-258. ACM Press, New York (2004)... 

Juan Michel Garcia Diaz was bom in Guadalajara, Jalisco, Mexico, on January 12, 1986, and lives in Villa de Alvarez, Colima, Mexico. He graduated in 2008 as a Computer Systems Engineer at the University of Colima, where he is now studying the Master of Computer Science. He is currently developing projects related to vehicular networks and multimodal interfaces. He is also interested in wireless communications networks. 

---