This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

ImageNet Auto-annotation with Segmentation Propagation

Matthieu Guillaumin, Daniel Kuettel, Vittorio Ferrari
International Journal of Computer Vision (IJCV)
2014, in press


ImageNet is a large-scale hierarchical database of object classes with millions of images. We propose to automatically populate it with pixelwise object-background segmentations, by leveraging existing manual annotations in the form of class labels and bounding-boxes. The key idea is to recursively exploit images segmented so far to guide the segmentation of new images. At each stage this propagation process expands into the images which are easiest to segment at that point in time, e.g. by moving to the semantically most related classes to those segmented so far. The propagation of segmentation occurs both (a) at the image level, by transferring existing segmentations to estimate the probability of a pixel to be foreground, and (b) at the class level, by jointly segmenting images of the same class and by importing the appearance models of classes that are already segmented. Through experiments on 577 classes and 500k images we show that our technique (i) annotates a wide range of classes with accurate segmentations; (ii) effectively exploits the hierarchical structure of ImageNet; (iii) scales efficiently, especially when implemented on superpixels; (iv) outperforms a baseline GrabCut initialized on the image center, as well as segmentation transfer from a fixed source pool and run independently on each target image. Moreover, our method also delivers state-of-the-art results on the recent iCoseg dataset for co-segmentation.

Link to publisher's page
  author = {Matthieu Guillaumin and Daniel Kuettel and Vittorio Ferrari},
  title = {ImageNet Auto-annotation with Segmentation Propagation},
  journal = {International Journal of Computer Vision (IJCV)},
  year = {2014},
  month = {},
  pages = {},
  volume = {},
  number = {},
  keywords = {Figure-ground segmentation; ImageNet; Knowledge transfer; Object localization; Large-scale computer vision},
  note = {in press}