A unified framework for content-aware view selection and planning through view importance

Abstract

In this paper we present new algorithms for Next-Best-View (NBV) planning and Image Selection (IS) aimed at image-based 3D reconstruction. In this context, NBV algorithms are needed to propose new unseen viewpoints to improve a partially reconstructed model, while IS algorithms are useful for selecting a subset of cameras from an unordered image collection before running an expensive dense reconstruction. Our methods are based on the idea of view importance: how important is a given viewpoint for a 3D reconstruction? We answer this by proposing a set of expressive quality features and formulate both problems as a search for views ranked by importance. Our methods are automatic and work directly on sparse Structure-from-Motion output. We can remove up to 90\% of the images and demonstrate improved speed at comparable reconstruction quality when compared with state of the art on multiple datasets.