Image segmentation is an important part in any computer vision framework. However, the transition from local low-level representations to useful structures and relations in the intermediate levels has turned out to be a truly difficult problem. This paper addresses the transition from low-level into intermediate-level vision, where the latter deals with producing a description of image and scene attributes in which more global relations are made explicit. The objective is to produce a hierarchical description of the scene in terms of clearly establishable relations between contours and vertices. We propose to combine a rich attributed contour representation with very general geometric contour relations. The implemented geometric relations, which are proximate, curvilinear, parallel and corner relations, allow to handle general man-made objects whose projected surfaces can be described by combinations of the defined relations, thus excluding amorphous objects such as clouds. The combination of rich image attributes and geometric relations allows to discriminate between strong and weak contour relations. Strong relations require that not only the geometrical constraints are met but also that the contour attributes (e.g. photometric) are in agreement. In this paper we describe the approach and show some preliminary results.