3D shape recovery under multiple viewpoints and single viewpoint

Abstract

This thesis introduces novel algorithms for 3D shape recovery under multiple viewpoints and single viewpoint. Surface of a 3D object is reconstructed by either graph-cuts using images under multiple viewpoints, depth from reflection under a fixed viewpoint, or depth from refraction under a fixed viewpoint.

The first part of this thesis revisits the graph-cuts based approach for solving the multi-view stereo problem and proposes a novel foreground / background energy. Unlike traditional graph-cuts based methods which focus on the photo-consistency energy, this thesis targets at deriving a robust and unbiased foreground / background energy which depends on data. It is shown that by using the proposed foreground / background energy, it is possible to recover the object surface from noisy depth maps even in the absence of the photo-consistency energy, which demonstrates the effectiveness of the proposed energy.

In the second part of this thesis, a novel method for shape recovery is proposed based on reflection of light using a spherical mirror. Unlike other existing methods which require the prior knowledge of the position and the radius of the spherical mirror, it is shown in this thesis that the object can be reconstructed up to an unknown scale using an unknown spherical mirror.

This thesis finally considers recovering object surfaces based on refraction of light and presents a novel depth from refraction method. A scene is captured several times by a fixed camera, with the first image (referred to as the direct image) captured directly by the camera and the others (referred to as the refracted images) by placing a transparent medium with two parallel planar faces between the scene and the camera. With a known pose and refractive index of the medium, a depth map of the scene is then recovered from the displacements of scene points in the images. Unlike traditional depth from refraction methods which require extra steps to estimate the pose and the refractive index of the medium, this thesis presents a novel method to estimate them from the direct and refracted images of the scene. It is shown that the pose of the medium can be recovered from one direct image and one refracted image. It is also shown that the refractive index of the medium can be recovered with a third image captured with the medium placed in a different pose.