Bi-level multi-column convolutional neural networks for facial landmark point detection

Abstract

We propose a bi-level Multi-column Convolutional Neural Networks (MCNNs) framework for face alignment. Global CNNs are used to roughly estimate the coordinates of all landmark points, and Local CNNs take patches sampled from the landmarks predicted by Global CNNs as input to predict the displacement between the ground truth and the landmark predicted by Global CNNs. The multi-column architecture leverages the findings that the optimal resolutions for different points are different. Further, the coordinates of all landmark and their displacement are simultaneously estimated in Global and Local CNNs, hence global shape constraints are naturally and implicitly imposed to make it very robust to significant variations in pose, expression, occlusion, and illumination. Extensive experiments demonstrate our method achieves state of the art performance for both image and video based face alignment on many publicly available datasets.