This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

2D Human Pose Estimation in TV Shows

V. Ferrari, M. Marin, and A. Zisserman
Statistical and Geometrical Approaches to Visual Motion Analysis
D. Cremers, B. Rosenhahn, A. Yuille, F. Schmidt, Ed.
Springer, 2009


The goal of this work is fully automatic 2D human pose estimation in unconstrained TV shows and feature films. Direct pose estimation on this uncontrolled material is often too difficult, especially when knowing nothing about the location, scale, pose, and appearance of the person, or even whether there is a person in the frame or not. We propose an approach that progressively reduces the search space for body parts, to greatly facilitate the task for the pose estimator. Moreover, when video is available, we propose methods for exploiting the temporal continuity of both appearance and pose for improving the estimation based on individual frames. The method is fully automatic and self-initializing, and explains the spatio-temporal volume covered by a person moving in a shot by soft-labeling every pixel as belonging to a particular body part or to the background. We demonstrate upper-body pose estimation by running our system on four episodes of the TV series 'Buffy the vampire slayer' (i.e. three hours of video). Our approach is evaluated quantitatively on several hundred video frames, based on ground-truth annotation of 2D poses (available at Finally, we present an application to full-body action recognition on the Weizmann dataset.

Download in pdf format
  title = {2D Human Pose Estimation in TV Shows},
  booktitle = {Statistical and Geometrical Approaches to Visual Motion Analysis},
  pages = {128-147},
  year = {2009},
  publisher = {Springer},
  keywords = {2D human pose estimation; human detection; action recognition}