This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Motion Capture Using Joint Skeleton Tracking and Surface Estimation

J. Gall, C. Stoll, E. de Aguiar, C. Theobalt, B. Rosenhahn and H.-P. Seidel
IEEE Conference on Computer Vision and Pattern Recognition


This paper proposes a method for capturing the performance of a human or an animal from a multi-view video sequence. Given an articulated template model and silhouettes from a multi-view image sequence, our approach recovers not only the movement of the skeleton, but also the possibly non-rigid temporal deformation of the 3D surface. While large scale deformations or fast movements are captured by the skeleton pose and approximate surface skinning, true small scale deformations or non-rigid garment motion are captured by fitting the surface to the silhouette. We further propose a novel optimization scheme for skeleton-based pose estimation that exploits the skeleton's tree structure to split the optimization problem into a local one and a lower dimensional global one. We show on various sequences that our approach can capture the 3D motion of animals and humans accurately even in the case of rapid movements and wide apparel like skirts.

Download in pdf format
  author = {J. Gall and C. Stoll and E. de Aguiar and C. Theobalt and B. Rosenhahn and H.-P. Seidel},
  title = {Motion Capture Using Joint Skeleton Tracking and Surface Estimation},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
  year = {2009},
  keywords = {}