This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

A 3-D Audio-Visual Corpus of Affective Communication

G. Fanelli and J. Gall and H. Romsdorfer and T. Weise and L. Van Gool
IEEE Transactions on Multimedia
Vol. 12, No. 6, pp. 591 - 598, October 2010


Communication between humans deeply relies on the capability of expressing and recognizing feelings. For this reason, research on human-machine interaction needs to focus on the recognition and simulation of emotional states, prerequisite of which is the collection of affective corpora. Currently available datasets still represent a bottleneck for the difficulties arising during the acquisition and labeling of affective data. In this work, we present a new audio-visual corpus for possibly the two most important modalities used by humans to communicate their emotional states, namely speech and facial expression in the form of dense dynamic 3-D face geometries. We acquire high-quality data by working in a controlled environment and resort to video clips to induce affective states. The annotation of the speech signal includes: transcription of the corpus text into the phonological representation, accurate phone segmentation, fundamental frequency extraction, and signal intensity estimation of the speech signals. We employ a real-time 3-D scanner to acquire dense dynamic facial geometries and track the faces throughout the sequences, achieving full spatial and temporal correspondences. The corpus is a valuable tool for applications like affective visual speech synthesis or view-independent facial expression recognition.

Link to publisher's page
Download in pdf format
  author = {G. Fanelli and J. Gall and H. Romsdorfer and T. Weise and L. Van Gool},
  title = {A 3-D Audio-Visual Corpus of Affective Communication},
  journal = {IEEE Transactions on Multimedia},
  year = {2010},
  month = {October},
  pages = {591 - 598},
  volume = {12},
  number = {6},
  keywords = {Audio-visual database, emotional speech, face tracking, visual speech modeling, 3-D face modeling}