File Download
  Links for fulltext
     (May Require Subscription)
Supplementary

postgraduate thesis: Mutual information-based depth estimation and 3D reconstruction for image-based rendering systems

TitleMutual information-based depth estimation and 3D reconstruction for image-based rendering systems
Authors
Advisors
Advisor(s):Chan, SCChang, C
Issue Date2012
PublisherThe University of Hong Kong (Pokfulam, Hong Kong)
Citation
Zhu, Z. [朱振宇]. (2012). Mutual information-based depth estimation and 3D reconstruction for image-based rendering systems. (Thesis). University of Hong Kong, Pokfulam, Hong Kong SAR. Retrieved from http://dx.doi.org/10.5353/th_b4832966
Abstract  Image-based rendering (IBR) is an emerging technology for rendering photo-realistic views of scenes from a collection of densely sampled images or videos. It provides a framework for developing revolutionary virtual reality and immersive viewing systems. There has been considerable progress recently in the capturing, storage and transmission of image-based representations. This thesis proposes two image-based rendering (IBR) systems for improving the viewing freedom and environmental modeling capability of conventional static IBR systems. The first system consists of a circular array with 13 still cameras (Canon 550D) for capturing ancient Chinese artifacts at high resolution. The second one is constructed by mounting a linear array of 8 video cameras (Sony HDR-TGIE) on an electrically controllable wheel chair with its motion being controllable manually or remotely through wireless local area network (LAN) by means of additional hardware circuitry.   Both systems support object-based rendering and 3D reconstruction capability and consist of two main components. 1) A novel view synthesis algorithm using a new segmentation and mutual information (MI)-based algorithm for dense depth map estimation, which relies on segmentation, local polynomial regression (LPR)-based depth map smoothing and MI-based matching algorithm to iteratively estimate the depth map. The method is very flexible and both semi-automatic and automatic segmentation methods can be employed. They rank fourth and sixth, respectively, in the Middlebury comparison of existing depth estimation methods. This allows high quality renderings of outdoor and indoor scenes with improved mobility/freedom to be obtained. This algorithm can also be extended to object tracking. Experimental results also show that the proposed MI-based algorithms are applicable to robust registration in noisy dynamic ultrasound images. 2) A new 3D reconstruction algorithm which utilizes sequential-structure-from-motion (S-SFM) technique and the dense depth maps estimated previously. It relies on a new iterative point cloud refinement algorithm based on Kalman filter (KF) for outlier removal and the segmentation-MI-based algorithm to further refine the correspondences and the projection matrices. The mobility of our system allows us to recover more conveniently 3D model of static objects from the improved point cloud using a new robust radial basis function (RBF)-based modeling algorithm to further suppress possible outliers and generate smooth 3D meshes of objects. Moreover, a new rendering technique named view dependent texture mapping is used to enhance the final rendering effect. Experimental results show that the proposed 3D reconstruction algorithm significantly reduces the adverse effect of the outliers and produces high quality renderings using view dependent texture mapping and the model reconstructed.   Overall, this study provides a framework for designing IBR systems with improved viewing freedom and ability to cope with moving and static objects in indoor and outdoor environment.   
DegreeDoctor of Philosophy
SubjectImage processing - Digital techniques.
Three-dimensional imaging.
Dept/ProgramElectrical and Electronic Engineering

 

DC FieldValueLanguage
dc.contributor.advisorChan, SC-
dc.contributor.advisorChang, C-
dc.contributor.authorZhu, Zhenyu-
dc.contributor.author朱振宇-
dc.date.issued2012-
dc.identifier.citationZhu, Z. [朱振宇]. (2012). Mutual information-based depth estimation and 3D reconstruction for image-based rendering systems. (Thesis). University of Hong Kong, Pokfulam, Hong Kong SAR. Retrieved from http://dx.doi.org/10.5353/th_b4832966-
dc.description.abstract  Image-based rendering (IBR) is an emerging technology for rendering photo-realistic views of scenes from a collection of densely sampled images or videos. It provides a framework for developing revolutionary virtual reality and immersive viewing systems. There has been considerable progress recently in the capturing, storage and transmission of image-based representations. This thesis proposes two image-based rendering (IBR) systems for improving the viewing freedom and environmental modeling capability of conventional static IBR systems. The first system consists of a circular array with 13 still cameras (Canon 550D) for capturing ancient Chinese artifacts at high resolution. The second one is constructed by mounting a linear array of 8 video cameras (Sony HDR-TGIE) on an electrically controllable wheel chair with its motion being controllable manually or remotely through wireless local area network (LAN) by means of additional hardware circuitry.   Both systems support object-based rendering and 3D reconstruction capability and consist of two main components. 1) A novel view synthesis algorithm using a new segmentation and mutual information (MI)-based algorithm for dense depth map estimation, which relies on segmentation, local polynomial regression (LPR)-based depth map smoothing and MI-based matching algorithm to iteratively estimate the depth map. The method is very flexible and both semi-automatic and automatic segmentation methods can be employed. They rank fourth and sixth, respectively, in the Middlebury comparison of existing depth estimation methods. This allows high quality renderings of outdoor and indoor scenes with improved mobility/freedom to be obtained. This algorithm can also be extended to object tracking. Experimental results also show that the proposed MI-based algorithms are applicable to robust registration in noisy dynamic ultrasound images. 2) A new 3D reconstruction algorithm which utilizes sequential-structure-from-motion (S-SFM) technique and the dense depth maps estimated previously. It relies on a new iterative point cloud refinement algorithm based on Kalman filter (KF) for outlier removal and the segmentation-MI-based algorithm to further refine the correspondences and the projection matrices. The mobility of our system allows us to recover more conveniently 3D model of static objects from the improved point cloud using a new robust radial basis function (RBF)-based modeling algorithm to further suppress possible outliers and generate smooth 3D meshes of objects. Moreover, a new rendering technique named view dependent texture mapping is used to enhance the final rendering effect. Experimental results show that the proposed 3D reconstruction algorithm significantly reduces the adverse effect of the outliers and produces high quality renderings using view dependent texture mapping and the model reconstructed.   Overall, this study provides a framework for designing IBR systems with improved viewing freedom and ability to cope with moving and static objects in indoor and outdoor environment.   -
dc.languageeng-
dc.publisherThe University of Hong Kong (Pokfulam, Hong Kong)-
dc.relation.ispartofHKU Theses Online (HKUTO)-
dc.rightsThe author retains all proprietary rights, (such as patent rights) and the right to use in future works.-
dc.rightsCreative Commons: Attribution 3.0 Hong Kong License-
dc.source.urihttp://hub.hku.hk/bib/B48329666-
dc.subject.lcshImage processing - Digital techniques.-
dc.subject.lcshThree-dimensional imaging.-
dc.titleMutual information-based depth estimation and 3D reconstruction for image-based rendering systems-
dc.typePG_Thesis-
dc.identifier.hkulb4832966-
dc.description.thesisnameDoctor of Philosophy-
dc.description.thesislevelDoctoral-
dc.description.thesisdisciplineElectrical and Electronic Engineering-
dc.description.naturepublished_or_final_version-
dc.identifier.doi10.5353/th_b4832966-
dc.date.hkucongregation2012-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats