Learning to Detect 3D Facial Landmarks via Heatmap Regression with Graph Convolutional Network

Yuan Wang, Min Cao, Zhenfeng Fan, Silong Peng

[AAAI-22] Main Track
Abstract: 3D facial landmark detection is extensively used in many research fields such as face registration, facial shape analysis, and face recognition. Most existing methods involve traditional features and 3D face models for the detection of landmarks, and their performances are limited by the hand-crafted intermediate process. In this paper, we propose a novel 3D facial landmark detection method, which directly locates the coordinates of landmarks from the 3D point cloud with a well-customized graph convolutional network. The graph convolutional network can learn geometric features adaptively for 3D facial landmark detection with the assistance of constructed 3D heatmaps, which are Gaussian functions of distances to each landmark on a 3D face. On this basis, we further develop a local surface unfolding and registration module to predict 3D landmarks from the heatmaps. We demonstrate experimentally that our method exceeds the existing approaches by a clear margin on BU-3DFE and FRGC datasets for landmark localization accuracy and stability. It also achieves high-precision results on a recent large-scale and high-quality FaceScape dataset. Generally, the proposed method forms the first baseline of the deep point cloud learning method for 3D facial landmark detection.

Introduction Video

Sessions where this paper appears

  • Poster Session 2

    Fri, February 25 12:45 AM - 2:30 AM (+00:00)
    Red 1
    Add to Calendar

  • Poster Session 12

    Mon, February 28 8:45 AM - 10:30 AM (+00:00)
    Red 1
    Add to Calendar