Tim Aebi

Master Thesis
Supervisors: Dr. Ajad Chhatkuli, Dr. Danda Pani Paudel, and Prof. Luc Van Gool

Unsupervised Deep-PnP for SLAM

A key element of visual Simultaneous Localization and Mapping (SLAM) pipelines is the problem of camera localization with respect to the map. In keypoint based approaches, this often boils down to solving a Perspective-n-Point (PnP) problem of world landmarks and their corresponding image pixels. Since outliers in the correspondence set are often unavoidable and have a fatal influence on localization accuracy, prior to feeding the correspondences to the PnP-solver usually an outlier rejection framework is employed. In this thesis we develop Deep-PnP, a robust, deep learning based outlier rejection frontend for the PnP problem, which in contrast to the commonly used Random Sampling and Consensus (RANSAC) algorithm has a constant execution time even for high outlier ratios. The proposed method can be trained in fully unsupervised manner. Furthermore, we compare Deep-PnP with RANSAC and show that our network based pipeline is competitive with RANSAC in terms of accuracy while being up to two orders of magnitude faster for higher outlier ratios due to its non-iterative nature. Finally we integrate Deep-PnP into a basic SLAM pipeline and present our preliminary results.