This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Dynamic 3D Scene Analysis from a Moving Vehicle

B. Leibe, N. Cornelis, K. Cornelis, L. Van Gool
IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07)
Minneapolis, USA, June 2007


In this paper, we present a system that integrates fully automatic scene geometry estimation, 2D object detection, 3D localization, trajectory estimation, and tracking for dynamic scene interpretation from a moving vehicle. Our sole input are two video streams from a calibrated stereo rig on top of a car. From these streams, we estimate Structurefrom- Motion (SfM) and scene geometry in real-time. In parallel, we perform multi-view/multi-category object recognition to detect cars and pedestrians in both camera images. Using the SfM self-localization, 2D object detections are converted to 3D observations, which are accumulated in a world coordinate frame. A subsequent tracking module analyzes the resulting 3D observations to find physically plausible spacetime trajectories. Finally, a global optimization criterion takes object-object interactions into account to arrive at accurate 3D localization and trajectory estimates for both cars and pedestrians. We demonstrate the performance of our integrated system on challenging real-world data showing car passages through crowded city areas.

Download in pdf format
  author = {B. Leibe and N. Cornelis and K. Cornelis and L. Van Gool},
  title = {Dynamic 3D Scene Analysis from a Moving Vehicle},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07)},
  year = {2007},
  month = {June},
  keywords = {}