File Download
There are no files associated with this item.
Links for fulltext
(May Require Subscription)
- Publisher Website: 10.1145/3588432.3591556
- Scopus: eid_2-s2.0-85167991151
- WOS: WOS:001117690500068
Supplementary
- Citations:
- Appears in Collections:
Conference Paper: Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild
Title | Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild |
---|---|
Authors | |
Keywords | Data-Driven Animation Facial Animation Facial Modeling Retargeting |
Issue Date | 23-Jul-2023 |
Abstract | We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild. Our approach, called Neural Face Rigging (NFR), holds three key properties: (i) NFR’s expression space maintains human-interpretable editing parameters for artistic controls; (ii) NFR is readily applicable to arbitrary facial meshes with different connectivity and expressions; (iii) NFR can encode and produce fine-grained details of complex expressions performed by arbitrary subjects. To the best of our knowledge, NFR is the first approach to provide realistic and controllable deformations of in-the-wild facial meshes, without the manual creation of blendshapes or correspondence. We design a deformation autoencoder and train it through a multi-dataset training scheme, which benefits from the unique advantages of two data sources: a linear 3DMM with interpretable control parameters as in FACS and 4D captures of real faces with fine-grained details. Through various experiments, we show NFR’s ability to automatically produce realistic and accurate facial deformations across a wide range of existing datasets and noisy facial scans in-the-wild, while providing artist-controlled, editable parameters. |
Persistent Identifier | http://hdl.handle.net/10722/333850 |
ISI Accession Number ID |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Qin, Dafei | - |
dc.contributor.author | Saito, Jun | - |
dc.contributor.author | Aigerman, Noam | - |
dc.contributor.author | Groueix, Thibault | - |
dc.contributor.author | Komura, Taku | - |
dc.date.accessioned | 2023-10-06T08:39:36Z | - |
dc.date.available | 2023-10-06T08:39:36Z | - |
dc.date.issued | 2023-07-23 | - |
dc.identifier.uri | http://hdl.handle.net/10722/333850 | - |
dc.description.abstract | <p> We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild. Our approach, called Neural Face Rigging (NFR), holds three key properties: (i) NFR’s expression space maintains human-interpretable editing parameters for artistic controls; (ii) NFR is readily applicable to arbitrary facial meshes with different connectivity and expressions; (iii) NFR can encode and produce fine-grained details of complex expressions performed by arbitrary subjects. To the best of our knowledge, NFR is the first approach to provide realistic and controllable deformations of in-the-wild facial meshes, without the manual creation of blendshapes or correspondence. We design a deformation autoencoder and train it through a multi-dataset training scheme, which benefits from the unique advantages of two data sources: a linear 3DMM with interpretable control parameters as in FACS and 4D captures of real faces with fine-grained details. Through various experiments, we show NFR’s ability to automatically produce realistic and accurate facial deformations across a wide range of existing datasets and noisy facial scans in-the-wild, while providing artist-controlled, editable parameters. <br></p> | - |
dc.language | eng | - |
dc.relation.ispartof | SIGGRAPH 2023 (06/08/2023-10/08/2023, Los Angeles) | - |
dc.subject | Data-Driven Animation | - |
dc.subject | Facial Animation | - |
dc.subject | Facial Modeling | - |
dc.subject | Retargeting | - |
dc.title | Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild | - |
dc.type | Conference_Paper | - |
dc.identifier.doi | 10.1145/3588432.3591556 | - |
dc.identifier.scopus | eid_2-s2.0-85167991151 | - |
dc.identifier.isi | WOS:001117690500068 | - |