This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Search for Publication

Year(s) from:  to 
Keywords (separated by spaces):

Here's looking at you, kid - Detecting people looking at each other in videos

M. Marin-Jimenez, A. Zisserman, and V. Ferrari
Proceedings of the British Machine Vision Conference
September 2011


The objective of this work is to determine if people are interacting in TV video by detecting whether they are looking at each other or not. We determine both the temporal period of the interaction and also spatially localize the relevant people. We make the following three contributions: (i) head pose estimation in unconstrained scenarios (TV video) using Gaussian Process regression; (ii) propose and evaluate several methods for assessing whether and when pairs of people are looking at each other in a video shot; and (iii) introduce new ground truth annotation for this task, extending the TV Human Interactions Dataset [22]. The peformance of the methods is evaluated on this dataset, which consists of 300 video clips extracted from TV shows. Despite the variety and difficulty of this video material, our best method obtains an average precision of 86.2%.

Download in pdf format
  author = {M. Marin-Jimenez and A. Zisserman and and V. Ferrari},
  title = {Here's looking at you, kid - Detecting people looking at each other in videos},
  booktitle = {Proceedings of the British Machine Vision Conference},
  year = {2011},
  month = {September},
  keywords = {video retrieval; human interaction}