Segmentation is in many cases the bottleneck when trying to use radiological image data in many clinically important applications as radiological diagnosis, monitoring, radiotherapy and surgical planning. Especially in case of large 3D medical data sets as obtained today by the routine use of 3D imaging methods like magnetic resonance imaging (MRI), computer tomography (CT) and ultrasound (US) the availability of efficient segmentation methods is a critical issue. While manual image segmentation is often regarded as a gold standard, its usage is not acceptable in some clinical situations. In some applications such as computer assisted neurosurgery or radiotherapy planning e.g., a large number of organs have to be identified in the radiological data sets. While a careful and time-consuming analysis may be acceptable for outlining complex pathological objects, no real justification for such a procedure can be found for the delineation of normal, healthy organs at risk. Delineation of organ boundaries is also necessary in various types of clinical studies, where the correlation between morphological changes and therapeutical actions or clinical diagnosis has to be analyzed. In order to get statistically significant results, a large number of data sets has to be segmented. For such applications manual segmentation becomes questionable not only because of the amount of work, but also with regard to the poor reproducibility of the results. Due to the above reasons, computer assisted segmentation is a very important problem to be solved in medical image analysis. During the past decades a huge body of literature has emerged, addressing all facets of the related scientific and algorithmic problems. A reasonably comprehensive review of all relevant efforts is clearly beyond the scope of this chapter. Instead, we just tried to analyze the underlying problems and principles and concisely summarize the most important research results, which have been achieved by several generations of PhD students at the Computer Vision Laboratory of the Swiss Federal Institute of Technology during the past 20 years.