T4.1 - AmI Use Case Interactive Knowledge in Action 01/2010 - 09/2011 Speech Communication Module Responsible partner: DFKI 1 F unctionality After that a fissioned multimodal presentation is produced and broadcasted to the Device Access Component and to the speech synthesis. For the speech recognition component we developed dedicated Grammar Based Language Models where grammar modules are matched to the corresponding dialogue situations. Fig. 2: Modular structure of the speech grammars oriented at the AMI situations 3 A pplication Fig. 1: Module highlighted in the software architecture of the AmI Case system The Speech Communication module (COMM) cares for the management of the communication between the system and the user. Therefore the module is responsible for the management of the speech-based direct interaction with the user. There are three main tasks: • Speech Input Interpretation: determination of “user intentions” (semantic result of interpretation) from recognition “hypothesis” • Dialog management: determining next steps in interaction with the user by: planning reaction, retrieving needed knowledge • Generation: generate presentation for multiple modalities 2 T echnical D escription The communication Module consists of the Communication Adapter Component and the following additional non IKSspecific multimodal dialog tools (Nuance Dragon Naturally Speaking, SVOX TTS, Ontology-Based Dialog Platform marketed by DFKI spin-off SemVox). In the specific the Communication Adapter Component permanently monitors the context development by watching at relevant broadcasted messages. If the user performs request (gesture, speech) a recognition hypothesis is generated and checked against the situational context in order to identify the expected task. Partners of IKS In the reiteration phase of the AMI case the VIE^2 Component from IKS Alpha has been integrated in the System in order to enhance the interaction capabilities with additional functionalities offered by enhanced content. The component has been exemplary used for the presentation of news content which is then semantically enhanced. The enhancement showed how a more interactive and flexible presentation could be reached by adding the VIE^2 IKS component. 4 I nter -M odule C ommunication The Speech Communication module receives speech input from the Device Input/Output Management module, retrieves needed content from the Knowledge Repository and passes results to the Context Management module. All data modifications are broadcasted to the Knowledge Repository via the Knowledge Access module. The multimodal presentation is passed to the Device Access Component and speech presentation directly to the speech synthesis system. 5 S trengths of M odule • Situation based speech recognition • Open microphone recognition • Output modalities and devices features aware Fission • Dialogical communication in an Ambient Intelligence Environment • Interplay with multiple input/output devices • Dialogical Interaction with content Task lead: Saarland University Sabine Janzen, [email protected] Contact: Massimo Romanelli, [email protected] http://www.iks-project.eu/