Data-Driven Sketch Interfaces for Modeling Objects Characterized by Curvilinear Structures

Professor Yu, Yizhou   (Principal Investigator (PI))

42

2019-06-28

2022-12-27

693000

Data-Driven Sketch Interfaces for Modeling Objects Characterized by Curvilinear Structures

Data-Driven Modeling, Deep Learning, Deformation, Face Modeling, Sketch-Based Modeling

Visualization & Graphics

Engineering (E)

17206218

General Research Fund (GRF)

2018

Completed

1) Investigate and further refine a novel sketching system for 3D face modeling. This system should have a labor-efficient sketching interface, and automatic 3D face model inference should be based on both deep learning techniques and constraint-based surface deformation techniques; 2) Develop an easy-to-use sketching system for interactive creation of personalized and photorealistic caricatures from photographs. Design a deep learning based method for inferring an exaggeration map for the underlying 3D face model recovered from an input photo. Build datasets for training and testing deep neural networks used in the sketching system; 3) Investigate a deep learning based solution to sketch-based modeling of generic objects characterized by curvilinear structures. A proposed deep learning based solution follows a coupled global-local network architecture, where a global network performs global structure inference while a local network performs local shape inference under the guidance of the global network.