This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Visual Recognition in RGB Images and Videos by Learning from RGB-D Data

Wen Li, Lin Chen, Dong Xu, and Luc Van Gool
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI)
2017, in press


In this work, we propose a framework for recognizing RGB images or videos by learning from RGB-D training data that contains additional depth information. We formulate this task as a new unsupervised domain adaptation (UDA) problem, in which we aim to take advantage of the additional depth features in the source domain and also cope with the data distribution mismatch between the source and target domains. To handle the domain distribution mismatch, we propose to learn an optimal projection matrix to map the samples from both domains into a common subspace such that the domain distribution mismatch can be reduced. Such projection matrix can be effectively optimized by exploiting different strategies. Moreover, we also use different ways to utilize the additional depth features. To simultaneously cope with the above two issues, we formulate a unified learning framework called domain adaptation from multi-view to single-view (DAM2S). By defining various forms of regularizers in our DAM2S framework, different strategies can be readily incorporated to learn robust SVM classifiers for classifying the target samples, and three methods are developed under our DAM2S framework. We conduct comprehensive experiments for object recognition, cross-dataset and cross-view action recognition, which demonstrate the effectiveness of our proposed methods for recognizing RGB images and videos by learning from RGB-D data.

Link to publisher's page
  author = {Wen Li and Lin Chen and Dong Xu and and Luc Van Gool},
  title = {Visual Recognition in RGB Images and Videos by Learning from RGB-D Data},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI)},
  year = {2017},
  month = {},
  pages = {},
  volume = {},
  number = {},
  keywords = {},
  note = {in press}