File Download
Supplementary

postgraduate thesis: Single view reconstruction of transparent, mirror and diffuse surfaces

TitleSingle view reconstruction of transparent, mirror and diffuse surfaces
Authors
Advisors
Advisor(s):Wong, KKY
Issue Date2018
PublisherThe University of Hong Kong (Pokfulam, Hong Kong)
Citation
Han, K. [韓鍇]. (2018). Single view reconstruction of transparent, mirror and diffuse surfaces. (Thesis). University of Hong Kong, Pokfulam, Hong Kong SAR.
Abstract3D reconstruction has been a fundamental problem in computer vision and has many applications. However, existing methods are mostly designed for diffuse surfaces under multiple viewpoints. This thesis tackles three reconstruction problems under a single view, namely, transparent object reconstruction, mirror surface reconstruction, and diffuse surface reconstruction. Besides, semantic correspondence, which is essential for not only 3D reconstruction but also image understanding, is also investigated in this thesis. In the first part of this thesis, a novel and practical approach is presented for transparent object reconstruction under a fixed viewpoint. A simple and handy setup is introduced to alter the incident light paths before light rays enter the object, followed by a surface recovery method based on reconstructing and triangulating such incident light paths. Our approach does not need to explicitly model the complex interactions of light as it travels through the object, assuming neither any parametric form for shape of the object nor exact number of refractions and reflections occur when light travels through the object. It can handle a transparent object with a complex structure, with an unknown and even inhomogeneous refractive index. This thesis then considers the problem of mirror surface reconstruction under a fixed viewpoint. We first derive an analytical solution to recover the camera projection matrix, and then optimize the camera projection matrix by minimizing reprojection errors with a cross-ratio formulation. The mirror surface is finally reconstructed based on the optimized cross-ratio constraint. The proposed method only needs reflection correspondences as input and removes the restrictive assumptions of known motions, $C^n$ continuity of the surface, and calibrated camera(s) that are being used by other existing methods. This greatly simplifies the challenging problem of mirror surface recovery. In the third part of this thesis, a novel self-calibration method is introduced for single view diffuse surface reconstruction using an unknown mirror sphere. We first derive an analytical solution to recover the focal length of the camera given its principal point, and then introduce a robust algorithm to estimate accurate principal point and the focal length of the camera. Besides, we also present a novel approach for estimating both principal point and focal length of the camera when only a single image of the sphere is available. With the estimated camera intrinsics, the sphere position and a scaled 3D scene object can be obtained. This thesis finally considers the problem of semantic correspondence estimation, which is crucial for 3D reconstruction as well as scene understanding. Most previous approaches to semantic correspondence focus on combining an effective spatial regularizer with hand-crafted features, or learning a correspondence model for appearance only. We proposed a convolutional neural network architecture, called SCNet, for learning a geometrically plausible model for semantic correspondence. SCNet uses region proposals as matching primitives, and explicitly incorporates geometric consistency.
DegreeDoctor of Philosophy
SubjectThree-dimensional imaging
Image reconstruction
Dept/ProgramComputer Science
Persistent Identifierhttp://hdl.handle.net/10722/261549

 

DC FieldValueLanguage
dc.contributor.advisorWong, KKY-
dc.contributor.authorHan, Kai-
dc.contributor.author韓鍇-
dc.date.accessioned2018-09-20T06:44:13Z-
dc.date.available2018-09-20T06:44:13Z-
dc.date.issued2018-
dc.identifier.citationHan, K. [韓鍇]. (2018). Single view reconstruction of transparent, mirror and diffuse surfaces. (Thesis). University of Hong Kong, Pokfulam, Hong Kong SAR.-
dc.identifier.urihttp://hdl.handle.net/10722/261549-
dc.description.abstract3D reconstruction has been a fundamental problem in computer vision and has many applications. However, existing methods are mostly designed for diffuse surfaces under multiple viewpoints. This thesis tackles three reconstruction problems under a single view, namely, transparent object reconstruction, mirror surface reconstruction, and diffuse surface reconstruction. Besides, semantic correspondence, which is essential for not only 3D reconstruction but also image understanding, is also investigated in this thesis. In the first part of this thesis, a novel and practical approach is presented for transparent object reconstruction under a fixed viewpoint. A simple and handy setup is introduced to alter the incident light paths before light rays enter the object, followed by a surface recovery method based on reconstructing and triangulating such incident light paths. Our approach does not need to explicitly model the complex interactions of light as it travels through the object, assuming neither any parametric form for shape of the object nor exact number of refractions and reflections occur when light travels through the object. It can handle a transparent object with a complex structure, with an unknown and even inhomogeneous refractive index. This thesis then considers the problem of mirror surface reconstruction under a fixed viewpoint. We first derive an analytical solution to recover the camera projection matrix, and then optimize the camera projection matrix by minimizing reprojection errors with a cross-ratio formulation. The mirror surface is finally reconstructed based on the optimized cross-ratio constraint. The proposed method only needs reflection correspondences as input and removes the restrictive assumptions of known motions, $C^n$ continuity of the surface, and calibrated camera(s) that are being used by other existing methods. This greatly simplifies the challenging problem of mirror surface recovery. In the third part of this thesis, a novel self-calibration method is introduced for single view diffuse surface reconstruction using an unknown mirror sphere. We first derive an analytical solution to recover the focal length of the camera given its principal point, and then introduce a robust algorithm to estimate accurate principal point and the focal length of the camera. Besides, we also present a novel approach for estimating both principal point and focal length of the camera when only a single image of the sphere is available. With the estimated camera intrinsics, the sphere position and a scaled 3D scene object can be obtained. This thesis finally considers the problem of semantic correspondence estimation, which is crucial for 3D reconstruction as well as scene understanding. Most previous approaches to semantic correspondence focus on combining an effective spatial regularizer with hand-crafted features, or learning a correspondence model for appearance only. We proposed a convolutional neural network architecture, called SCNet, for learning a geometrically plausible model for semantic correspondence. SCNet uses region proposals as matching primitives, and explicitly incorporates geometric consistency.-
dc.languageeng-
dc.publisherThe University of Hong Kong (Pokfulam, Hong Kong)-
dc.relation.ispartofHKU Theses Online (HKUTO)-
dc.rightsThe author retains all proprietary rights, (such as patent rights) and the right to use in future works.-
dc.rightsThis work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.-
dc.subject.lcshThree-dimensional imaging-
dc.subject.lcshImage reconstruction-
dc.titleSingle view reconstruction of transparent, mirror and diffuse surfaces-
dc.typePG_Thesis-
dc.description.thesisnameDoctor of Philosophy-
dc.description.thesislevelDoctoral-
dc.description.thesisdisciplineComputer Science-
dc.description.naturepublished_or_final_version-
dc.date.hkucongregation2018-
dc.identifier.mmsid991044040578103414-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats