Full Text:   <991>

Summary:  <678>

CLC number: TP319

On-line Access: 2018-02-06

Received: 2017-04-16

Revision Accepted: 2017-08-09

Crosschecked: 2017-12-20

Cited: 0

Clicked: 2461

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Feng Liu

http://orcid.org/0000-0002-6625-0593

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2017 Vol.18 No.12 P.1978-1990

http://doi.org/10.1631/FITEE.1700253


On 3D face reconstruction via cascaded regression in shape space


Author(s):  Feng Liu, Dan Zeng, Jing Li, Qi-jun Zhao

Affiliation(s):  College of Computer Science, Sichuan University, Chengdu 610065, China

Corresponding email(s):   qjzhao@scu.edu.cn

Key Words:  3D face reconstruction, Cascaded regressor, Shape space, Real-time


Feng Liu, Dan Zeng, Jing Li, Qi-jun Zhao. On 3D face reconstruction via cascaded regression in shape space[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(12): 1978-1990.

@article{title="On 3D face reconstruction via cascaded regression in shape space",
author="Feng Liu, Dan Zeng, Jing Li, Qi-jun Zhao",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="18",
number="12",
pages="1978-1990",
year="2017",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.1700253"
}

%0 Journal Article
%T On 3D face reconstruction via cascaded regression in shape space
%A Feng Liu
%A Dan Zeng
%A Jing Li
%A Qi-jun Zhao
%J Frontiers of Information Technology & Electronic Engineering
%V 18
%N 12
%P 1978-1990
%@ 2095-9184
%D 2017
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.1700253

TY - JOUR
T1 - On 3D face reconstruction via cascaded regression in shape space
A1 - Feng Liu
A1 - Dan Zeng
A1 - Jing Li
A1 - Qi-jun Zhao
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 18
IS - 12
SP - 1978
EP - 1990
%@ 2095-9184
Y1 - 2017
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.1700253


Abstract: 
Cascaded regression has been recently applied to reconstruct 3D faces from single 2D images directly in shape space, and has achieved state-of-the-art performance. We investigate thoroughly such cascaded regression based 3D face reconstruction approaches from four perspectives that are not well been studied: (1) the impact of the number of 2D landmarks; (2) the impact of the number of 3D vertices; (3) the way of using standalone automated landmark detection methods; (4) the convergence property. To answer these questions, a simplified cascaded regression based 3D face reconstruction method is devised. This can be integrated with standalone automated landmark detection methods and reconstruct 3D face shapes that have the same pose and expression as the input face images, rather than normalized pose and expression. An effective training method is also proposed by disturbing the automatically detected landmarks. Comprehensive evaluation experiments have been carried out to compare to other 3D face reconstruction methods. The results not only deepen the understanding of cascaded regression based 3D face reconstruction approaches, but also prove the effectiveness of the proposed method.

形状空间下基于级联回归的三维人脸重建

概要:最近,级联回归方法在基于二维图像的三维人脸重建中取得了良好的效果。本文主要探究了基于级联回归三维人脸重建中四个未解决的重要问题:(1)二维特征点数量的影响;(2)三维点云数量的影响;(3)自动二维特征点的检测方法;(4)算法的收敛。本文专门设计了一个简单有效的基于形状空间的级联回归三维重建算法。该算法有效利用各类独立的自动二维特征点检测算法,且重建的三维人脸具有和输入图像相同的姿态和表情。同时在训练中,本文设计了一种随机扰动检二维特征点位置信息的策略,使得算法更有效且更鲁棒。最后,通过大量的对比实验证明了该算法不仅有着良好的精度和计算效率,而且对加深级联回归三维重建方法的理解有着启发意义。

关键词:三维人脸重建;级联回归;形状空间;实时

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Abiantun, R., Prabhu, U., Savvides, M., 2014. Sparse feature extraction for pose-tolerant face recognition. IEEE Trans. Patt. Anal. Mach. Intell., 36(10):2061-2073.

[2]Aldrian, O., Smith, W.A.P., 2013. Inverse rendering of faces with a 3D morphable model. IEEE Trans. Patt. Anal. Mach. Intell., 35(5):1080-1093.

[3]Bagdanov, A.D., Del Bimbo, A., Masi, I., 2011. The florence 2D/3D hybrid face dataset. Workshop on Human Gesture and Behavior Understanding, p.79-80.

[4]Barron, J.T., Malik, J., 2012. Shape, albedo, and illumination from a single image of an unknown object. IEEE Conf. on Computer Vision and Pattern Recognition, p.334-341.

[5]Bas, A., Smith, W.A.P., Bolkart, T., et al., 2016. Fitting a 3D morphable model to edges: a comparison between hard and soft correspondences. IEEE Asian Conf. on Computer Vision, p.377-391.

[6]Blanz, V., Vetter, T., 1999. A morphable model for the synthesis of 3D faces. Proc. SIGGRAPH, p.187-194.

[7]Blanz, V., Vetter, T., 2003. Face recognition based on fitting a 3D morphable model. IEEE Trans. Patt. Anal. Mach. Intell., 25(9):1063-1074.

[8]Booth, J., Roussos, A., Zafeiriou, S., et al., 2016. A 3D morphable model learnt from 10,000 faces. IEEE Conf. on Computer Vision and Pattern Recognition, p.5543-5552.

[9]Cao, C., Hou, Q.M., Zhou, K., 2014a. Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph., 33(4):43.1-43.10.

[10]Cao, C., Weng, Y.L., Zhou, S., et al., 2014b. Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph., 20(3):413-425.

[11]Cao, C., Wu, H.Z., Weng, Y.L., et al., 2016. Real-time facial animation with image-based dynamic avatars. ACM Trans. Graph., 35(4):126.1-126.12.

[12]Chu, B., Romdhani, S., Chen, L.M., 2014. 3D-aided face recognition robust to expression and pose variations. IEEE Conf. on Computer Vision and Pattern Recognition, p.1907-1914.

[13]Han, H., Jain, A.K., 2012. 3D face texture modeling from uncalibrated frontal and profile images. Int. Conf. on Biometrics: Theory, Applications and Systems, p.223-230.

[14]Horn, B.K.P., Brooks, M.J., 1989. Shape from Shading. MIT Press, Cambridge, MA, USA.

[15]Hu, J.L., Lu, J.W., Tan, Y.P., 2014. Discriminative deep metric learning for face verification in the wild. IEEE Conf. on Computer Vision and Pattern Recognition, p.1875-1882.

[16]Jourabloo, A., Liu, X.M., 2015. Pose-invariant 3D face alignment. IEEE Int. Conf. on Computer Vision, p.3694-3702.

[17]Jourabloo, A., Liu, X.M., 2016. Large-pose face alignment via CNN-based dense 3D model fitting. IEEE Conf. on Computer Vision and Pattern Recognition, p.4188-4196.

[18]Jourabloo, A., Liu, X.M., 2017. Pose-invariant face alignment via CNN-based dense 3D model fitting. Int. J. Comput. Vis., 4:1-17.

[19]Kazemi, V., Sullivan, J., 2014. One millisecond face alignment with an ensemble of regression trees. IEEE Conf. on Computer Vision and Pattern Recognition, p.1867-1874.

[20]Kemelmacher-Shlizerman, I., Basri, R., 2011. 3D face reconstruction from a single image using a single reference face shape. IEEE Trans. Patt. Anal. Mach. Intell., 33(2):394-405.

[21]Köstinger, M., Wohlhart, P., Roth, P.M., et al., 2011. Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. IEEE Int. Conf. on Computer Vision Workshops, p.2144-2151.

[22]Li, J., Long, S.Q., Zeng, D., et al., 2015. Example-based 3D face reconstruction from uncalibrated frontal and profile images. IEEE Int. Conf. on Biometrics, p.193-200.

[23]Li, X., Xu, Y.D., Lv, Q., et al., 2016. Affine-transformation parameters regression for face alignment. IEEE Signal Process. Lett., 23(1):55-59.

[24]Liu, F., Zeng, D., Zhao, Q.J., et al., 2016. Joint face alignment and 3D face reconstruction. European Conf. on Computer Vision, p.545-560.

[25]Paysan, P., Knothe, R., Amberg, B., et al., 2009. A 3D face model for pose and illumination invariant face recognition. IEEE Conf. on Advanced Video and Signal-based Surveillance, p.296-301.

[26]Ren, J.F., Jiang, X.D., Yuan, J.S., 2016. Face and facial expressions recognition and analysis. Context Aware Human-Robot and Human-Agent Interaction, p.3-29.

[27]Romdhani, S., Vetter, T., 2005. Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. IEEE Conf. on Computer Vision and Pattern Recognition, p.986-993.

[28]Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., et al., 2013. 300 faces in-the-wild challenge: the first facial landmark localization challenge. IEEE Int. Conf. on Computer Vision Workshops, p.397-403.

[29]Sun, Y., Wang, X.G., Tang, X.O., 2013. Deep convolutional network cascade for facial point detection. IEEE Conf. on Computer Vision and Pattern Recognition, p.3476-3483.

[30]Suwajanakorn, S., Kemelmacher-Shlizerman, I., Seitz, S.M., 2014. Total moving face reconstruction. European Conf. on Computer Vision, p.796-812.

[31]Xiong, X.H., de la Torre, F., 2013. Supervised descent method and its applications to face alignment. IEEE Conf. on Computer Vision and Pattern Recognition, p.532-539.

[32]Yin, L.J., Wei, X.Z., Sun, Y., et al., 2006. A 3D facial expression database for facial behavior research. IEEE Int. Conf. on Automatic Face and Gesture Recognition, p.211-216.

[33]Zeng, D., Zhao, Q.J., Long, S.Q., et al., 2017. Examplar coherent 3D face reconstruction from forensic mugshot database. Image Vis. Comput., 58:193-203.

[34]Zhang, Z.P., Luo, P., Chen, C.L., et al., 2014. Facial landmark detection by deep multi-task learning. European Conf. on Computer Vision, p.94-108.

[35]Zhou, X.W., Leonardos, S., Hu, X.Y., et al., 2015. 3D shape estimation from 2D landmarks: a convex relaxation approach. IEEE Conf. on Computer Vision and Pattern Recognition, p.4447-4455.

[36]Zhu, S.Z., Li, C., Chen, C.L., et al., 2015. Face alignment by coarse-to-fine shape searching. IEEE Conf. on Computer Vision and Pattern Recognition, p.4998-5006.

[37]Zhu, X.X., Ramanan, D., 2012. Face detection, pose estimation, and landmark localization in the wild. IEEE Conf. on Computer Vision and Pattern Recognition, p.2879-2886.

[38]Zhu, X.Y., Yi, D., Lei, Z., et al., 2014. Robust 3D morphable model fitting by sparse SIFT flow. IEEE Int. Conf. on Pattern Recognition, p.4044-4049.

[39]Zhu, X.Y., Lei, Z., Yan, J.J., et al., 2015. High-fidelity pose and expression normalization for face recognition in the wild. IEEE Conf. on Computer Vision and Pattern Recognition, p.787-796.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE