Abstract: Cascaded regression has been recently applied to reconstruct 3D faces from single 2D images directly in shape space, and has achieved state-of-the-art performance. We investigate thoroughly such cascaded regression based 3D face reconstruction approaches from four perspectives that are not well been studied: (1) the impact of the number of 2D landmarks; (2) the impact of the number of 3D vertices; (3) the way of using standalone automated landmark detection methods; (4) the convergence property. To answer these questions, a simplified cascaded regression based 3D face reconstruction method is devised. This can be integrated with standalone automated landmark detection methods and reconstruct 3D face shapes that have the same pose and expression as the input face images, rather than normalized pose and expression. An effective training method is also proposed by disturbing the automatically detected landmarks. Comprehensive evaluation experiments have been carried out to compare to other 3D face reconstruction methods. The results not only deepen the understanding of cascaded regression based 3D face reconstruction approaches, but also prove the effectiveness of the proposed method.

形状空间下基于级联回归的三维人脸重建

[1]Abiantun, R., Prabhu, U., Savvides, M., 2014. Sparse feature extraction for pose-tolerant face recognition. IEEE Trans. Patt. Anal. Mach. Intell., 36(10):2061-2073.

[2]Aldrian, O., Smith, W.A.P., 2013. Inverse rendering of faces with a 3D morphable model. IEEE Trans. Patt. Anal. Mach. Intell., 35(5):1080-1093.

[3]Bagdanov, A.D., Del Bimbo, A., Masi, I., 2011. The florence 2D/3D hybrid face dataset. Workshop on Human Gesture and Behavior Understanding, p.79-80.

[4]Barron, J.T., Malik, J., 2012. Shape, albedo, and illumination from a single image of an unknown object. IEEE Conf. on Computer Vision and Pattern Recognition, p.334-341.

[5]Bas, A., Smith, W.A.P., Bolkart, T., et al., 2016. Fitting a 3D morphable model to edges: a comparison between hard and soft correspondences. IEEE Asian Conf. on Computer Vision, p.377-391.

[6]Blanz, V., Vetter, T., 1999. A morphable model for the synthesis of 3D faces. Proc. SIGGRAPH, p.187-194.

[7]Blanz, V., Vetter, T., 2003. Face recognition based on fitting a 3D morphable model. IEEE Trans. Patt. Anal. Mach. Intell., 25(9):1063-1074.

[8]Booth, J., Roussos, A., Zafeiriou, S., et al., 2016. A 3D morphable model learnt from 10,000 faces. IEEE Conf. on Computer Vision and Pattern Recognition, p.5543-5552.

[9]Cao, C., Hou, Q.M., Zhou, K., 2014a. Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph., 33(4):43.1-43.10.

[10]Cao, C., Weng, Y.L., Zhou, S., et al., 2014b. Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph., 20(3):413-425.

[11]Cao, C., Wu, H.Z., Weng, Y.L., et al., 2016. Real-time facial animation with image-based dynamic avatars. ACM Trans. Graph., 35(4):126.1-126.12.

[12]Chu, B., Romdhani, S., Chen, L.M., 2014. 3D-aided face recognition robust to expression and pose variations. IEEE Conf. on Computer Vision and Pattern Recognition, p.1907-1914.

[13]Han, H., Jain, A.K., 2012. 3D face texture modeling from uncalibrated frontal and profile images. Int. Conf. on Biometrics: Theory, Applications and Systems, p.223-230.

[14]Horn, B.K.P., Brooks, M.J., 1989. Shape from Shading. MIT Press, Cambridge, MA, USA.

[15]Hu, J.L., Lu, J.W., Tan, Y.P., 2014. Discriminative deep metric learning for face verification in the wild. IEEE Conf. on Computer Vision and Pattern Recognition, p.1875-1882.

[16]Jourabloo, A., Liu, X.M., 2015. Pose-invariant 3D face alignment. IEEE Int. Conf. on Computer Vision, p.3694-3702.

[17]Jourabloo, A., Liu, X.M., 2016. Large-pose face alignment via CNN-based dense 3D model fitting. IEEE Conf. on Computer Vision and Pattern Recognition, p.4188-4196.

[18]Jourabloo, A., Liu, X.M., 2017. Pose-invariant face alignment via CNN-based dense 3D model fitting. Int. J. Comput. Vis., 4:1-17.

[19]Kazemi, V., Sullivan, J., 2014. One millisecond face alignment with an ensemble of regression trees. IEEE Conf. on Computer Vision and Pattern Recognition, p.1867-1874.

[20]Kemelmacher-Shlizerman, I., Basri, R., 2011. 3D face reconstruction from a single image using a single reference face shape. IEEE Trans. Patt. Anal. Mach. Intell., 33(2):394-405.

[21]Köstinger, M., Wohlhart, P., Roth, P.M., et al., 2011. Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. IEEE Int. Conf. on Computer Vision Workshops, p.2144-2151.

[22]Li, J., Long, S.Q., Zeng, D., et al., 2015. Example-based 3D face reconstruction from uncalibrated frontal and profile images. IEEE Int. Conf. on Biometrics, p.193-200.

[23]Li, X., Xu, Y.D., Lv, Q., et al., 2016. Affine-transformation parameters regression for face alignment. IEEE Signal Process. Lett., 23(1):55-59.

[24]Liu, F., Zeng, D., Zhao, Q.J., et al., 2016. Joint face alignment and 3D face reconstruction. European Conf. on Computer Vision, p.545-560.

[25]Paysan, P., Knothe, R., Amberg, B., et al., 2009. A 3D face model for pose and illumination invariant face recognition. IEEE Conf. on Advanced Video and Signal-based Surveillance, p.296-301.

[26]Ren, J.F., Jiang, X.D., Yuan, J.S., 2016. Face and facial expressions recognition and analysis. Context Aware Human-Robot and Human-Agent Interaction, p.3-29.

[27]Romdhani, S., Vetter, T., 2005. Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. IEEE Conf. on Computer Vision and Pattern Recognition, p.986-993.

[28]Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., et al., 2013. 300 faces in-the-wild challenge: the first facial landmark localization challenge. IEEE Int. Conf. on Computer Vision Workshops, p.397-403.

[29]Sun, Y., Wang, X.G., Tang, X.O., 2013. Deep convolutional network cascade for facial point detection. IEEE Conf. on Computer Vision and Pattern Recognition, p.3476-3483.

[30]Suwajanakorn, S., Kemelmacher-Shlizerman, I., Seitz, S.M., 2014. Total moving face reconstruction. European Conf. on Computer Vision, p.796-812.

[31]Xiong, X.H., de la Torre, F., 2013. Supervised descent method and its applications to face alignment. IEEE Conf. on Computer Vision and Pattern Recognition, p.532-539.

[32]Yin, L.J., Wei, X.Z., Sun, Y., et al., 2006. A 3D facial expression database for facial behavior research. IEEE Int. Conf. on Automatic Face and Gesture Recognition, p.211-216.

[33]Zeng, D., Zhao, Q.J., Long, S.Q., et al., 2017. Examplar coherent 3D face reconstruction from forensic mugshot database. Image Vis. Comput., 58:193-203.

[34]Zhang, Z.P., Luo, P., Chen, C.L., et al., 2014. Facial landmark detection by deep multi-task learning. European Conf. on Computer Vision, p.94-108.

[35]Zhou, X.W., Leonardos, S., Hu, X.Y., et al., 2015. 3D shape estimation from 2D landmarks: a convex relaxation approach. IEEE Conf. on Computer Vision and Pattern Recognition, p.4447-4455.

[36]Zhu, S.Z., Li, C., Chen, C.L., et al., 2015. Face alignment by coarse-to-fine shape searching. IEEE Conf. on Computer Vision and Pattern Recognition, p.4998-5006.

[37]Zhu, X.X., Ramanan, D., 2012. Face detection, pose estimation, and landmark localization in the wild. IEEE Conf. on Computer Vision and Pattern Recognition, p.2879-2886.

[38]Zhu, X.Y., Yi, D., Lei, Z., et al., 2014. Robust 3D morphable model fitting by sparse SIFT flow. IEEE Int. Conf. on Pattern Recognition, p.4044-4049.

[39]Zhu, X.Y., Lei, Z., Yan, J.J., et al., 2015. High-fidelity pose and expression normalization for face recognition in the wild. IEEE Conf. on Computer Vision and Pattern Recognition, p.787-796.