This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Functional Categorization of Objects using Real-time Markerless Motion Capture

J. Gall, A. Fossati and L. Van Gool
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Colorado Springs, CO, USA, June 2011


Unsupervised categorization of objects is a fundamental problem in computer vision. While appearance-based methods have become popular recently, other important cues like functionality are largely neglected. Motivated by psychological studies giving evidence that human demonstration has a facilitative effect on categorization in infancy, we propose an approach for object categorization from depth video streams. To this end, we have developed a method for capturing human motion in real-time. The captured data is then used to temporally segment the depth streams into actions. The set of segmented actions are then categorized in an unsupervised manner, through a novel descriptor for motion capture data that is robust to subject variations. Furthermore, we automatically localize the object that is manipulated within a video segment, and categorize it using the corresponding action. For evaluation, we have recorded adataset that comprises depth data with registered video sequences for 6 subjects, 13 action classes, and 174 object manipulations.

Download in pdf format
  author = {J. Gall and A. Fossati and L. Van Gool},
  title = {Functional Categorization of Objects using Real-time Markerless Motion Capture},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2011},
  month = {June},
  pages = {1969-1976},
  keywords = {}