In this report we approach the problem of extracting planes from aerial images. The reason for extracting planar patches is that man-made objects (houses, buildings) are usually constructed out of connected planes. However, we need 3D data to extract the planes, therefore, we present a framework which consists of a segment stereo-matching and a coplanar grouping algorithm. A number of methods for edge-based stereo-matching rely on extracting 2D edges from images and then match them. The main drawback with these methods is when a contour in one image is occluded or broken, then it cannot be matched. Another approach makes use of an edge template that moves along the epipolar line to find the correspondences. We present a variant of the latter kind which we denote segment stereo-matching. The algorithm proceeds in two steps. First, for each 2D edge extracted from one image, we search the correspondences in the other images. This is performed by maximizing an edginess measure along the epipolar line. Secondly, we merge all the matches and keep only the consistent ones. The merging process is achieved with geometric and photometric constraints. This method is well suited for finding correspondences even in the case of broken or occluded contours. Then, we present a scheme for robust extraction of planar patches from a set of 3D edges spoiled with many gross errors. The method proceeds in three steps: exploration, merging, and extension. During the exploration we find an initial set of hypotheses of planes. The exploration is done using a deterministic search driven by 2D geometric relations. The merging process selects the hypotheses describing the same plane and merges them into one instantiation to have a unique description of this plane. Then, each hypothesis is extended by searching for all the segments that lie on its plane. Finally, we apply this framework on aerial images and show some results of reconstructed scenes in 3D and segmentation of theses scenes into planar patches.