REPLICATE aims at the creation of powerful tools to produce 3D models, made available to many users.
The project's consortium consists of a team research institutions and SMEs. Smartphones are used to take photos, from which the 3D models are extracted. This 3D extraction combines the increasing computational power of phones with that of the cloud. REPLICATE builds an entire 3D creation suite, including a structure-from-motion 3D model generation part, model editing tools, and ways to combine models into exciting 3D worlds.
ETH is developing solutions for the acquisition of 3D shapes, turning smartphone photos into 3D models through a combination of local computation on the mobile devices and remote computation in a computer cloud. Successful modeling is supported by an interface that provides the user with a partial 3D reconstruction on-the-fly and advises the user where it might be the best to take further images to improve the model. Special focus is given to the actual distribution of the computation between the mobile and the cloud, which may depend on many factors including the quality of current Internet connection, the computational power of the mobile processor, and current battery level.
The joint mobile-cloud solution will facilitate an advanced level of object understanding as it is integrated into the 3D acquisition process. This will yield better 3D models with enhanced usability. ETH is also investigating the semantic analysis of models, e.g., segmentation of windows, doors, etc. in order to fill in unavoidable holes in the 3D point cloud models. To this end, methods extracting geometrical primitives like straight lines or planes from point clouds will be developed as well.