This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Combining Image-level and Segment-level Models for Automatic Annotation

Daniel Kuettel, Matthieu Guillaumin, Vittorio Ferrari
The 18th International Conference on MultiMedia Modeling


For the task of assigning labels to an image to summarize its contents, many early attempts use segment-level information and try to determine which parts of the images correspond to which labels. Best performing methods use global image similarity and nearest neighbor techniques to transfer labels from training images to test images. However, global methods cannot localize the labels in the images, unlike segment-level methods. Also, they cannot take advantage of training images that are only locally similar to a test image. We propose several ways to combine recent image-level and segment-level techniques to predict both image and segment labels jointly. We cast our experimental study in an unified framework for both image-level and segment-level annotation tasks. On three challenging datasets, our joint prediction of image and segment labels outperforms either prediction alone on both tasks. This confirms that the two levels offer complementary information.

Download in pdf format
  author = {Daniel Kuettel and Matthieu Guillaumin and Vittorio Ferrari},
  title = {Combining Image-level and Segment-level Models for Automatic Annotation},
  booktitle = {The 18th International Conference on MultiMedia Modeling},
  year = {2012},
  publisher = {Springer},
  keywords = {image auto-annotation, image region labelling, keyword-based image retrieval}