The automatic reconstruction of 3D city models from high resolution aerial images is still an area of active research. An automation of the reconstruction task is highly desirable, since the manual reconstruction requires expensive instruments, highly qualified personnel, and is very time consuming. Even worse, the task demands high concentration from the human operator and thus is error-prone. On the other hand there is a broad area of planning tasks, where 3D city models already show their usefulness:
  • Line-of-Sight network planning
  • town planning
  • Computer Aided Architectural Design
  • rescue and intervention scenarios for police and fire department
  • micro-climate simulations
  • (virtual) tourism
  • navigation systems
  • ...
Clearly, as soon as cheap, reliable, and up-to-date 3D city models become available on the market, a multitude of new interesting application areas will evolve.


In a previous ETH-project (AMOBE I) it has been successfully shown, that an automatic reconstruction of isolated buildings in suburban scenes is possible, if the location of the building in the image is known. In the AMOBE II project, the given task will be extended to densely built-up urban areas. This causes qualitatively and quantitatively new difficulties stemming from the more complicated roof shapes and the typical situation of buildings located close or contiguous to each other. Therefore the following problems have to be solved:
  • automatic detection of settled areas
  • reconstruction of contiguous buildings
  • integration of building models
  • detection of errors occurring during the reconstruction process

The project partners Institute of Geodesy and Photogrammetry work in the automatic detection of man-made objects - especially houses and roads - in densely build-up urban areas. They also provide the input datasets for the project.
The Computer Vision Group focuses on the 3D reconstruction and spatial grouping. For 3D building reconstruction straight line segments at roof edges need to be matched between corresponding views. Solving the correspondence problem is not straightforward. To overcome the geometric ambiguities at the stereo matching step, an improved matching algorithm was developed.

Line Segment Stereo Matching

We presented the stereo matching algorithm on the ISPRS conference 2000 in Amsterdam. The presentation and the paper are available online. Our strategy to systematically reduce the complexity of the correspondence problem is based on an iterative scheme. After each reduction step a more sophisticated elimination procedure can be applied, since the number of remaining combinations gets smaller after each step. As a novelty, also the color information of the input images is used through the matching process.

Some results for different levels of reduction are listed below. By clicking on the appropriate thumbnail a simple 3D-wireframe viewer is started. Starting the applet can take some time over a slow network connection, especially for the large models.

Straight line segments at roof edges are matched between overlapping views. This task is not straightforward. Mainly due to the weak epipolar constraint, there is a huge number of possible pairings between extracted edges in different views. A calculation of all geometrically possible 3D line reconstructions therefore yields mostly futile results. 3421
To overcome the geometric ambiguities, we take into account the color of regions flanking the extracted line segments. By comparing the integrated color distribution in the flanking regions of putative pairs, we determine a statistical chromatic similarity measure. This step reduces the possible combinations to typically 30 percent. 1245
A further significant reduction is achieved by computing a cross-correlation based chromatic similarity measure, exploiting the pixel-wise one-to-one correspondence induced by epipolar geometry. 443
For the remaining pairs the 3D information is calculated, taking into account a third view via the trifocal tensor. The optimal line reconstruction is then determined by bundle adjustment. 63
Also the use of the trifocal tensor does not allow a total disambiguation of the correspondence problem. But again, by comparing the color of regions flanking the extracted line segments the number of wrong matches is reduced. 57

Roof patch hypotheses

For roof plane detection, roof patch delineation and the final patch grouping different plane hypotheses have to be instantiated and verified. To deal with the inevitably incomplete data obtained from the previous matching step we are currently working on a probabilistic approach. The approach is intended to allow the integration of data with different levels of uncertainty in a consistent manner.


For further information please address to Stephan Scholze or Prof. Luc Van Gool.