Temporal Coordination of Facial Expressions and Head Movements in First Encounter Dialogues

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningpeer review

Abstract

This paper deals with the temporal coordination between facial expressions and co-occurring head movements in a multimodal corpus of first encounter conversations. In particular, we look at how the onset of facial expressions is coordinated with the first overlapping head movement, in other words which of the two modalities precedes the other and why. We find and discuss statistical main effects on the temporal delays between the two behaviours due to individual variation, type of head movement, and the communicative function of the multimodal signal. In particular, the analysis shows that when speakers give feedback, their facial expression becomes visible before the head starts to move, especially in the case of negative comments associated with frowning or scowling. The opposite is true when the multimodal signal is used as a comment to the speaker's own speech. The motivation for the analysis is to shed light on a less studied aspect of multimodal communication -- an aspect that is relevant to the generation of natural multimodal expressions in ECAs.
OriginalsprogEngelsk
TitelProceedings of the LREC 2018 Workshop "LB-ILR2018 and MMC2018 Joint Workshop" : Language and Body in Real Life
RedaktørerHanae Koiso, Patrizia Paggio
Antal sider5
ForlagEuropean Language Resources Association
Publikationsdato2018
Sider40-44
ISBN (Elektronisk)979-10-95546-16-0
StatusUdgivet - 2018

Citationsformater