Abstract: When obtaining three-dimensional (3D) face point cloud data based on structured light, factors related to the environment, occlusion, and illumination intensity lead to holes in the collected data, which affect subsequent recognition. In this study, we propose a hole-filling method based on stereo-matching technology combined with a B-spline. The algorithm uses phase information acquired during raster projection to locate holes in the point cloud, simultaneously extracting boundary point cloud sets. By registering the face point cloud data using the stereo-matching algorithm and the data collected using the raster projection method, some supplementary information points can be obtained at the holes. The shape of the B-spline curve can then be roughly described by a few key points, and the control points are put into the hole area as key points for iterative calculation of surface reconstruction. Simulations using smooth ceramic cups and human face models showed that our model can accurately reproduce details and accurately restore complex shapes on the test surfaces. Simulation results indicated the robustness of the method, which is able to fill holes on complex areas such as the inner side of the nose without a prior model. This approach also effectively supplements the hole information, and the patched point cloud is closer to the original data. This method could be used across a wide range of applications requiring accurate facial recognition.

基于双目立体匹配和B样条的三维人脸点云孔洞修补算法

[1]BendelsGH, SchnabelR, KleinR, 2006. Detecting holes in point set surfaces. J WSCG, 14:89-96.

[2]BlackJAJr., GargeshaM, KaholK, et al., 2016. Framework for performance evaluation of face recognition algorithms. SPIE 4862:163-174.

[3]BuchinK, van KreveldM, MeijerH, et al., 2009. On planar supports for hypergraphs. Proc 17^th Int Conf on Graph Drawing, p.345-356.

[4]CarrJC, BeatsonRK, CherrieJB, et al., 2001. Reconstruction and representation of 3D objects with radial basis functions. Proc 28^th Annual Conf on Computer Graphics and Interactive Techniques, p.67-76.

[5]ChenH, MaSW, NuechterA, 2016. Non-synchronous point cloud algorithm for 3D reconstruction based on laser scanning and SFM. Chin J Sci Instrum, 37(5):1148-1157 (in Chinese).

[6]ChuiCK, LaiMJ, 2000. Filling polygonal holes using C¹ cubic triangular spline patches. Comput Aided Geom Des, 17(4):297-307.

[7]FloaterMS, ReimersM, 2001. Meshless parameterization and surface reconstruction. Comput Aided Geom Des, 18(2): 77-92.

[8]FurukawaY, PonceJ, 2010. Accurate, dense, and robust multiview stereopsis. IEEE Trans Patt Anal Mach Intell, 32(8):1362-1376.

[9]GilaniZS, MianA, 2016. Towards large-scale 3D face recognition. Proc Int Conf on Digital Image Computing: Techniques and Applications, p.1-8.

[10]GoeseleM, CurlessB, SeitzSM, 2006. Multi-view stereo revisited. Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition, p.2402-2409.

[11]HuangXS, ZhangJ, FanLX, et al., 2017. A systematic approach for cross-source point cloud registration by preserving macro and micro structures. IEEE Trans Image Process, 26(7):3261-3276.

[12]HuangY, DaFP, 2019. Registration algorithm for point cloud based on normalized cross-correlation. IEEE Access, 7:137136-137146.

[13]HuangY, DaFP, TaoHJ, 2015. An automatic registration algorithm for point cloud based on feature extraction. Chin J Lasers, 42(3):308002 (in Chinese).

[14]JeongY, BokY, KimJS, et al., 2011. Complementation of cameras and lasers for accurate 6D SLAM: from correspondences to bundle adjustment. Proc IEEE Int Conf on Robotics and Automation, p.3581-3588.

[15]JunY, 2005. A piecewise hole filling algorithm in reverse engineering. Comput-Aided Des, 37(2):263-270.

[16]KurlinV, 2014. A fast and robust algorithm to count topologically persistent holes in noisy clouds. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1458-1463.

[17]LiuK, ZhouCH, WeiSB, et al., 2014. Optimized stereo matching in binocular three-dimensional measurement system using structured light. Appl Opt, 53(26):6083-6090.

[18]LiuYJ, WangM, ZhangHL, et al., 2016. Strategy of classification and repairing for hole of incomplete point clouds based on fuzzy inference. J Comput Theor Nanosci, 13 (11):8227-8233.

[19]NguyenVS, TrinhTH, TranMH, 2015. Hole boundary detection of a surface of 3D point clouds. Proc Int Conf on Advanced Computing and Applications, p.124-129.

[20]OrriolsX, BinefaX, 2003. Finding breaking curves in 3D surface. Proc 1^st Iberian Conf on Pattern Recognition and Image Analysis, p.681-688.

[21]O’TooleAJ, AnXB, DunlopJ, et al., 2012. Comparing face recognition algorithms to humans on challenging tasks. ACM Trans Appl Percept, 9(4):16.

[22]PanYH, 2019. On visual knowledge. Front Inform Technol Electron Eng, 20(8):1021-1025.

[23]PanYH, 2021. Miniaturized five fundamental issues about visual knowledge. Front Inform Technol Electron Eng, 22(5):615-618.

[24]PanchettiM, PernotJP, VéronP, 2010. Towards recovery of complex shapes in meshes using digital images for reverse engineering applications. Comput-Aided Des, 42(8):693-707.

[25]PernotJP, MoraruG, VéronP, 2007. Repairing triangle meshes built from scanned point cloud. J Eng Des, 18(5): 459-473.

[26]QuinsatY, LartigueC, 2015. Filling holes in digitized point cloud using a morphing-based approach to preserve volume characteristics. Int J Adv Manuf Technol, 81(1-4): 411-421.

[27]Russell, StuartJ, NorvigP, 2010. Artificial intelligence: a modern approach. Appl Mech Mater, 263(5):2829-2833.

[28]SchafferM, GrosseM, HarendtB, et al., 2011. High-speed three-dimensional shape measurements of objects with laser speckles and acousto-optical deflection. Opt Lett, 36(16):3097-3099.

[29]ShiLM, GuoFS, HuZY, 2011. An improved PMVS through scene geometric information. Acta Autom Sin, 37(5):560-568 (in Chinese).

[30]StoneEE, SkubicM, 2015. Fall detection in homes of older adults using the Microsoft Kinect. IEEE J Biomed Health Inform, 19(1):290-301.

[31]WangJN, OliveiraMM, 2007. Filling holes on locally smooth surfaces reconstructed from point clouds. Image Vis Comput, 25(1):103-113.

[32]ZhengEL, WuCC, 2015. Structure from motion using structure-less resection. Proc IEEE Int Conf on Computer Vision, p.2075-2083.