This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Face Animation Based on Observed 3D Speech Dynamics

G. A. Kalberer and L. Van Gool
Procs. of The Fourteenth IEEE Conf. on Computer Animation (CA'01)
Seoul, Korea 2001


Realistic face animation is especially hard as we are all experts in the perception and interpretation of face dynamics. One approach is to simulate facial anatomy. Alternatively, animation can be based on first observing the visible 3D dynamics, extracting the basic modes, and putting these together according to the required performance. This is the strategy followed by the paper, which focuses on speech. The approach follows a kind of bootstrap procedure. First, 3D shape statistics are learned from a talking face with a relatively small number of markers. A 3D reconstruction is produced at temporal intervals of 1/25 seconds. A topological mask of the lower half of the face is fitted to the motion. Principal component analysis (PCA) of the mask shapes reduces the dimension of the mask shape space. The result is two-fold. On the one hand, the face can be animated, in our case it can be made to speak new sentences. On the other hand, face dynamics can be tracked in 3D without markers for performance capture.

Download in pdf format
  author = {G. A. Kalberer and L. Van Gool},
  title = {Face Animation Based on Observed 3D Speech Dynamics},
  booktitle = {Procs. of The Fourteenth IEEE Conf. on Computer Animation (CA'01)},
  year = {2001},
  pages = {20--27},
  editor = {H-S. Ko},
  publisher = {IEEE Computer Society},
  keywords = {face animation, pca}