CLIN 2005 Abstracts
  • Multimodal Treatment of Presuppositions Interpretation in Natural Dialog Discourse
    Olga Vybornova (UCL-TELE, Batiment Stevin)
    Benoit Macq (Belgium Laboratoire de Telecommunications et Teledetection, Universite Catholique de Louvain (UCL))
    A challenge of multimodal approach in natural conversational interaction is to observe correlation between transfer of presupposed (background, implicit) information in the dialog and emotions. The goal of the work is to develop a system of multimodal natural dialog discourse analysis based on emotion detection (by speech prosody and facial expressions) and on detection and interpretation of presuppositions.

    We fix the moments of breach of presuppositions and check change of emotions at these moments - facial expression, mimicry (graphics), as well as intonation, voice loudness, pitch, timbre, speech tempo (acoustics). And on the contrary - change of emotions might mean that some inconsistency occurred in the dialog communication, that there is a mismatch in presuppositions and that this situation should be corrected.

    We argue that the process of presuppositions interpretation is probabilistic, we assign a priori probabilities to potential outcomes of presupposed information interpretation. Here multimodal emotion detection is going to be used as an error-correction code and as a weighting process, it will help to refine and redistribute probabilities in the natural language processing part. And vice versa - in real-life dialogs, the context helps to correctly evaluate the interaction-dependent emotions thus proving the importance of integrating dialog information to improve emotion recognition.

    Dynamic Bayesian networks will be used to fuse multimodal information coming from natural language analysis and emotion detection. The Bayesian networks will be trained on real-life data - video recordings of mutlimodal dialogs.