Recent work on early vision such as image segmentation, image denoising, stereo matching, and optical flow uses Markov Random Fields. Although this formulation yields an NP-hard energy minimization problem, good heuristics have been developed based on graph cuts and belief propagation. Nevertheless both approaches still require tens of seconds to solve stereo problems on recent PCs. Such running times are impractical for optical flow and many image segmentation and denoising problems and we review recent techniques for speeding them up. Moreover, we show how to reduce the computational complexity of belief propagation by applying the Four Color Theorem to limit the maximum number of labels in the underlying image segmentation to at most four. We show that this provides substantial speed improvements for large inputs, and this for a variety of vision problems, while maintaining competitive result quality.