Applications using pedestrian detection in street scene require both high speed and quality. Maximal speed is reached when exploiting the geometric information provided by stereo cameras. Yet, extracting useful information at speeds higher than 100 Hz is a non-trivial task. We propose a method to estimate the ground-obstacles boundary (and its distance), without computing a depth map. By properly parametrizing the search space in the image plane we improve the algorithmic performance, and reach speeds of 200 Hz on a desktop CPU. When connected with a state of the art GPU objects detector, we reach high quality detections at the record speed of 165 Hz.