File Download

There are no files associated with this item.

  Links for fulltext
     (May Require Subscription)
Supplementary

Article: GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation

TitleGeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation
Authors
KeywordsThree-dimensional displays
Surface reconstruction
Estimation
Image reconstruction
Computer architecture
Issue Date2020
PublisherIEEE. The Journal's web site is located at http://www.computer.org/tpami
Citation
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, Epub 2020-09-01, p. 1-1 How to Cite?
AbstractIn this paper, we propose a geometric neural network with edge-aware refinement (GeoNet++) to jointly predict both depth and surface normal maps from a single image. Building on top of two-stream CNNs, GeoNet++ captures the geometric relationships between depth and surface normals with the proposed depth-to-normal and normal-to-depth modules. In particular, the “depth-to-normal” module exploits the least square solution of estimating surface normals from depth to improve their quality, while the “normal-to-depth” module refines the depth map based on the constraints on surface normals through kernel regression. Boundary information is exploited via an edge-aware refinement module. GeoNet++ effectively predicts depth and surface normals with high 3D consistency and sharp boundaries resulting in better reconstructed 3D scenes. Note that GeoNet++ is generic and can be used in other depth/normal prediction frameworks to improve 3D reconstruction quality and pixel-wise accuracy of depth and surface normals. Furthermore, we propose a new 3D geometric metric (3DGM) for evaluating depth prediction in 3D. In contrast to current metrics that focus on evaluating pixel-wise error/accuracy, 3DGM measures whether the predicted depth can reconstruct high-quality 3D surface normals. This is a more natural metric for many 3D application domains. Our experiments on NYUD-V2 and KITTI demonstrate the effectiveness of our approach.
Persistent Identifierhttp://hdl.handle.net/10722/287658
ISSN
2020 Impact Factor: 16.389
2020 SCImago Journal Rankings: 3.811

 

DC FieldValueLanguage
dc.contributor.authorQi, X-
dc.contributor.authorLiu, Z-
dc.contributor.authorLiao, R-
dc.contributor.authorTorr, P-
dc.contributor.authorUrtasun, R-
dc.contributor.authorJia, J-
dc.date.accessioned2020-10-05T12:01:20Z-
dc.date.available2020-10-05T12:01:20Z-
dc.date.issued2020-
dc.identifier.citationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, Epub 2020-09-01, p. 1-1-
dc.identifier.issn0162-8828-
dc.identifier.urihttp://hdl.handle.net/10722/287658-
dc.description.abstractIn this paper, we propose a geometric neural network with edge-aware refinement (GeoNet++) to jointly predict both depth and surface normal maps from a single image. Building on top of two-stream CNNs, GeoNet++ captures the geometric relationships between depth and surface normals with the proposed depth-to-normal and normal-to-depth modules. In particular, the “depth-to-normal” module exploits the least square solution of estimating surface normals from depth to improve their quality, while the “normal-to-depth” module refines the depth map based on the constraints on surface normals through kernel regression. Boundary information is exploited via an edge-aware refinement module. GeoNet++ effectively predicts depth and surface normals with high 3D consistency and sharp boundaries resulting in better reconstructed 3D scenes. Note that GeoNet++ is generic and can be used in other depth/normal prediction frameworks to improve 3D reconstruction quality and pixel-wise accuracy of depth and surface normals. Furthermore, we propose a new 3D geometric metric (3DGM) for evaluating depth prediction in 3D. In contrast to current metrics that focus on evaluating pixel-wise error/accuracy, 3DGM measures whether the predicted depth can reconstruct high-quality 3D surface normals. This is a more natural metric for many 3D application domains. Our experiments on NYUD-V2 and KITTI demonstrate the effectiveness of our approach.-
dc.languageeng-
dc.publisherIEEE. The Journal's web site is located at http://www.computer.org/tpami-
dc.relation.ispartofIEEE Transactions on Pattern Analysis and Machine Intelligence-
dc.rightsIEEE Transactions on Pattern Analysis and Machine Intelligence. Copyright © IEEE.-
dc.rights©20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.-
dc.subjectThree-dimensional displays-
dc.subjectSurface reconstruction-
dc.subjectEstimation-
dc.subjectImage reconstruction-
dc.subjectComputer architecture-
dc.titleGeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation-
dc.typeArticle-
dc.identifier.emailQi, X: xjqi@eee.hku.hk-
dc.identifier.authorityQi, X=rp02666-
dc.description.naturelink_to_subscribed_fulltext-
dc.identifier.doi10.1109/TPAMI.2020.3020800-
dc.identifier.pmid32870785-
dc.identifier.hkuros315446-
dc.identifier.volumeEpub 2020-09-01-
dc.identifier.spage1-
dc.identifier.epage1-
dc.publisher.placeUnited States-
dc.identifier.issnl0162-8828-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats