This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

A Mobile Vision System for Robust Multi-Person Tracking

A. Ess, B. Leibe, K. Schindler, and L. Van Gool
IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08)
Anchorage, USA, June 2008


We present a mobile vision system for multi-person tracking in busy environments. Specifically, the system integrates continuous visual odometry computation with tracking-by-detection in order to track pedestrians in spite of frequent occlusions and egomotion of the camera rig. To achieve reliable performance under real-world conditions, it has long been advocated to extract and combine as much visual information as possible. We propose a way to closely integrate the vision modules for visual odometry, pedestrian detection, depth estimation, and tracking. The integration naturally leads to several cognitive feedback loops between the modules. Among others, we propose a novel feedback connection from the object detector to visual odometry which utilizes the semantic knowledge of detection to stabilize localization. Feedback loops always carry the danger that erroneous feedback from one module is amplified and causes the entire system to become instable. We therefore incorporate automatic failure detection and recovery, allowing the system to continue when a module becomes unreliable. The approach is experimentally evaluated on several long and difficult video sequences from busy inner-city locations. Our results show that the proposed integration makes it possible to deliver stable tracking performance in scenes of previously infeasible complexity.

Download in pdf format
  author = {A. Ess and B. Leibe and K. Schindler and and L. Van Gool},
  title = {A Mobile Vision System for Robust Multi-Person Tracking},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08)},
  year = {2008},
  month = {June},
  keywords = {}