File Download
  Links for fulltext
     (May Require Subscription)
Supplementary

Article: DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling

TitleDeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling
Authors
Issue Date2017
PublisherAssociation for Computing Machinery, Inc. The Journal's web site is located at http://tog.acm.org
Citation
ACM Transactions on Graphics, 2017, v. 36 n. 4, p. 126:1-12 How to Cite?
AbstractFace modeling has been paid much attention in the field of visual computing. There exist many scenarios, including cartoon characters, avatars for social media, 3D face caricatures as well as face-related art and design, where low-cost interactive face modeling is a popular approach especially among amateur users. In this paper, we propose a deep learning based sketching system for 3D face and caricature modeling. This system has a labor-efficient sketching interface, that allows the user to draw freehand imprecise yet expressive 2D lines representing the contours of facial features. A novel CNN based deep regression network is designed for inferring 3D face models from 2D sketches. Our network fuses both CNN and shape based features of the input sketch, and has two independent branches of fully connected layers generating independent subsets of coefficients for a bilinear face representation. Our system also supports gesture based interactions for users to further manipulate initial face models. Both user studies and numerical results indicate that our sketching system can help users create face models quickly and effectively. A significantly expanded face database with diverse identities, expressions and levels of exaggeration is constructed to promote further research and evaluation of face modeling techniques.
Persistent Identifierhttp://hdl.handle.net/10722/243519
ISSN
2017 Impact Factor: 4.384
2015 SCImago Journal Rankings: 2.552
ISI Accession Number ID

 

DC FieldValueLanguage
dc.contributor.authorHan, X-
dc.contributor.authorGao, C-
dc.contributor.authorYu, Y-
dc.date.accessioned2017-08-25T02:55:54Z-
dc.date.available2017-08-25T02:55:54Z-
dc.date.issued2017-
dc.identifier.citationACM Transactions on Graphics, 2017, v. 36 n. 4, p. 126:1-12-
dc.identifier.issn0730-0301-
dc.identifier.urihttp://hdl.handle.net/10722/243519-
dc.description.abstractFace modeling has been paid much attention in the field of visual computing. There exist many scenarios, including cartoon characters, avatars for social media, 3D face caricatures as well as face-related art and design, where low-cost interactive face modeling is a popular approach especially among amateur users. In this paper, we propose a deep learning based sketching system for 3D face and caricature modeling. This system has a labor-efficient sketching interface, that allows the user to draw freehand imprecise yet expressive 2D lines representing the contours of facial features. A novel CNN based deep regression network is designed for inferring 3D face models from 2D sketches. Our network fuses both CNN and shape based features of the input sketch, and has two independent branches of fully connected layers generating independent subsets of coefficients for a bilinear face representation. Our system also supports gesture based interactions for users to further manipulate initial face models. Both user studies and numerical results indicate that our sketching system can help users create face models quickly and effectively. A significantly expanded face database with diverse identities, expressions and levels of exaggeration is constructed to promote further research and evaluation of face modeling techniques.-
dc.languageeng-
dc.publisherAssociation for Computing Machinery, Inc. The Journal's web site is located at http://tog.acm.org-
dc.relation.ispartofACM Transactions on Graphics-
dc.rightsACM Transactions on Graphics. Copyright © Association for Computing Machinery, Inc.-
dc.rightsThis work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.-
dc.titleDeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling-
dc.typeArticle-
dc.identifier.emailYu, Y: yzyu@cs.hku.hk-
dc.identifier.authorityYu, Y=rp01415-
dc.description.naturepublished_or_final_version-
dc.identifier.doi10.1145/3072959.3073629-
dc.identifier.hkuros273679-
dc.identifier.volume36-
dc.identifier.issue4-
dc.identifier.spage126:1-
dc.identifier.epage126:12-
dc.identifier.isiWOS:000406432100094-
dc.publisher.placeUnited States-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats