This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Harmony Potentials: Fusing Global and Local Scale for Semantic Image Segmentation

Xavier Boix, Josep M. Gonfaus, Joost Van de Weijer, Andrew D. Bagdanov, Joan Serrat, and Jordi Gonzàlez
International Journal of Computer Vision (IJCV)
Vol. 91, No. 1, pp. 83-102, January 2012


Abstract The Hierarchical Conditional Random Field (HCRF) model have been successfully applied to a number of image labeling problems, including image segmentation. However, existing HCRF models of image segmentation do not allow multiple classes to be assigned to a single region, which limits their ability to incorporate contextual information across multiple scales. At higher scales in the image, this representation yields an oversimplified model since multiple classes can be reasonably expected to appear within large regions. This simplified model particularly limits the impact of information at higher scales. Since class-label information at these scales is usually more reliable than at lower, noisier scales, neglecting this information is undesirable. To address these issues, we propose a new consistency potential for image labeling problems, which we call the harmony potential. It can encode any possible combination of labels, penalizing only unlikely combinations of classes. We also propose an effective sampling strategy over this expanded label set that renders tractable the underlying optimization problem. Our approach obtains state-of-the-art results on two challenging, standard benchmark datasets for semantic image segmentation: PASCAL VOC 2010, and MSRC-21.

Download in pdf format
  author = {Xavier Boix and Josep M. Gonfaus and Joost Van de Weijer and Andrew D. Bagdanov and Joan Serrat and and Jordi Gonzàlez},
  title = {Harmony Potentials: Fusing Global and Local Scale for Semantic Image Segmentation},
  journal = {International Journal of Computer Vision (IJCV)},
  year = {2012},
  month = {January},
  pages = {83-102},
  volume = {91},
  number = {1},
  keywords = {}