This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Weakly Supervised Semantic Segmentation with a Multi-image Model

A. Vezhnevets, V. Ferrari, and J. Buhmann
Proceedings of the International Conference on Computer Vision
November 2011


We propose a novel method for weakly supervised se- mantic segmentation. Training images are labeled only by the classes they contain, not by their location in the image. On test images instead, the method predicts a class label for every pixel. Our main innovation is a multi-image model (MIM) - a graphical model for recovering the pixel labels of the training images. The model connects superpixels from all training images in a data-driven fashion, based on their appearance similarity. For generalizing to new test images we integrate them into MIM using a learned multiple kernel metric, instead of learning conventional classifiers on the recovered pixel labels. We also introduce an “objectness” potential, that helps separating objects (e.g. car, dog, hu- man) from background classes (e.g. grass, sky, road). In ex- periments on the MSRC 21 dataset and the LabelMe subset of [18], our technique outperforms previous weakly super- vised methods and achieves accuracy comparable with fully supervised methods.

Download in pdf format
  author = {A. Vezhnevets and V. Ferrari and and J. Buhmann},
  title = {Weakly Supervised Semantic Segmentation with a Multi-image Model},
  booktitle = {Proceedings of the International Conference on Computer Vision},
  year = {2011},
  month = {November},
  keywords = {}