Full Text:   <4095>

Summary:  <318>

CLC number: TP391.4

On-line Access: 2022-03-22

Received: 2020-09-28

Revision Accepted: 2021-04-15

Crosschecked: 0000-00-00

Cited: 0

Clicked: 5884

Citations:  Bibtex RefMan EndNote GB/T7714




-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2022 Vol.23 No.3 P.398-408


Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline

Author(s):  Yuan HUANG, Feipeng DA

Affiliation(s):  School of Automation, Southeast University, Nanjing 210096, China; more

Corresponding email(s):   whhbb@163.com

Key Words:  Three-dimensional (3D) point cloud, Hole filling, Stereo matching, B-spline

Yuan HUANG, Feipeng DA. Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(3): 398-408.

@article{title="Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline",
author="Yuan HUANG, Feipeng DA",
journal="Frontiers of Information Technology & Electronic Engineering",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline
%A Feipeng DA
%J Frontiers of Information Technology & Electronic Engineering
%V 23
%N 3
%P 398-408
%@ 2095-9184
%D 2022
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2000508

T1 - Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline
A1 - Yuan HUANG
A1 - Feipeng DA
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 23
IS - 3
SP - 398
EP - 408
%@ 2095-9184
Y1 - 2022
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2000508

When obtaining three-dimensional (3D) face point cloud data based on structured light, factors related to the environment, occlusion, and illumination intensity lead to holes in the collected data, which affect subsequent recognition. In this study, we propose a hole-filling method based on stereo-matching technology combined with a b-spline. The algorithm uses phase information acquired during raster projection to locate holes in the point cloud, simultaneously extracting boundary point cloud sets. By registering the face point cloud data using the stereo-matching algorithm and the data collected using the raster projection method, some supplementary information points can be obtained at the holes. The shape of the b-spline curve can then be roughly described by a few key points, and the control points are put into the hole area as key points for iterative calculation of surface reconstruction. Simulations using smooth ceramic cups and human face models showed that our model can accurately reproduce details and accurately restore complex shapes on the test surfaces. Simulation results indicated the robustness of the method, which is able to fill holes on complex areas such as the inner side of the nose without a prior model. This approach also effectively supplements the hole information, and the patched point cloud is closer to the original data. This method could be used across a wide range of applications requiring accurate facial recognition.




Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]BendelsGH, SchnabelR, KleinR, 2006. Detecting holes in point set surfaces. J WSCG, 14:89-96.

[2]BlackJAJr., GargeshaM, KaholK, et al., 2016. Framework for performance evaluation of face recognition algorithms. SPIE 4862:163-174.

[3]BuchinK, van KreveldM, MeijerH, et al., 2009. On planar supports for hypergraphs. Proc 17th Int Conf on Graph Drawing, p.345-356.

[4]CarrJC, BeatsonRK, CherrieJB, et al., 2001. Reconstruction and representation of 3D objects with radial basis functions. Proc 28th Annual Conf on Computer Graphics and Interactive Techniques, p.67-76.

[5]ChenH, MaSW, NuechterA, 2016. Non-synchronous point cloud algorithm for 3D reconstruction based on laser scanning and SFM. Chin J Sci Instrum, 37(5):1148-1157 (in Chinese).

[6]ChuiCK, LaiMJ, 2000. Filling polygonal holes using C1 cubic triangular spline patches. Comput Aided Geom Des, 17(4):297-307.

[7]FloaterMS, ReimersM, 2001. Meshless parameterization and surface reconstruction. Comput Aided Geom Des, 18(2): 77-92.

[8]FurukawaY, PonceJ, 2010. Accurate, dense, and robust multiview stereopsis. IEEE Trans Patt Anal Mach Intell, 32(8):1362-1376.

[9]GilaniZS, MianA, 2016. Towards large-scale 3D face recognition. Proc Int Conf on Digital Image Computing: Techniques and Applications, p.1-8.

[10]GoeseleM, CurlessB, SeitzSM, 2006. Multi-view stereo revisited. Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition, p.2402-2409.

[11]HuangXS, ZhangJ, FanLX, et al., 2017. A systematic approach for cross-source point cloud registration by preserving macro and micro structures. IEEE Trans Image Process, 26(7):3261-3276.

[12]HuangY, DaFP, 2019. Registration algorithm for point cloud based on normalized cross-correlation. IEEE Access, 7:137136-137146.

[13]HuangY, DaFP, TaoHJ, 2015. An automatic registration algorithm for point cloud based on feature extraction. Chin J Lasers, 42(3):308002 (in Chinese).

[14]JeongY, BokY, KimJS, et al., 2011. Complementation of cameras and lasers for accurate 6D SLAM: from correspondences to bundle adjustment. Proc IEEE Int Conf on Robotics and Automation, p.3581-3588.

[15]JunY, 2005. A piecewise hole filling algorithm in reverse engineering. Comput-Aided Des, 37(2):263-270.

[16]KurlinV, 2014. A fast and robust algorithm to count topologically persistent holes in noisy clouds. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1458-1463.

[17]LiuK, ZhouCH, WeiSB, et al., 2014. Optimized stereo matching in binocular three-dimensional measurement system using structured light. Appl Opt, 53(26):6083-6090.

[18]LiuYJ, WangM, ZhangHL, et al., 2016. Strategy of classification and repairing for hole of incomplete point clouds based on fuzzy inference. J Comput Theor Nanosci, 13 (11):8227-8233.

[19]NguyenVS, TrinhTH, TranMH, 2015. Hole boundary detection of a surface of 3D point clouds. Proc Int Conf on Advanced Computing and Applications, p.124-129.

[20]OrriolsX, BinefaX, 2003. Finding breaking curves in 3D surface. Proc 1st Iberian Conf on Pattern Recognition and Image Analysis, p.681-688.

[21]O’TooleAJ, AnXB, DunlopJ, et al., 2012. Comparing face recognition algorithms to humans on challenging tasks. ACM Trans Appl Percept, 9(4):16.

[22]PanYH, 2019. On visual knowledge. Front Inform Technol Electron Eng, 20(8):1021-1025.

[23]PanYH, 2021. Miniaturized five fundamental issues about visual knowledge. Front Inform Technol Electron Eng, 22(5):615-618.

[24]PanchettiM, PernotJP, VéronP, 2010. Towards recovery of complex shapes in meshes using digital images for reverse engineering applications. Comput-Aided Des, 42(8):693-707.

[25]PernotJP, MoraruG, VéronP, 2007. Repairing triangle meshes built from scanned point cloud. J Eng Des, 18(5): 459-473.

[26]QuinsatY, LartigueC, 2015. Filling holes in digitized point cloud using a morphing-based approach to preserve volume characteristics. Int J Adv Manuf Technol, 81(1-4): 411-421.

[27]Russell, StuartJ, NorvigP, 2010. Artificial intelligence: a modern approach. Appl Mech Mater, 263(5):2829-2833.

[28]SchafferM, GrosseM, HarendtB, et al., 2011. High-speed three-dimensional shape measurements of objects with laser speckles and acousto-optical deflection. Opt Lett, 36(16):3097-3099.

[29]ShiLM, GuoFS, HuZY, 2011. An improved PMVS through scene geometric information. Acta Autom Sin, 37(5):560-568 (in Chinese).

[30]StoneEE, SkubicM, 2015. Fall detection in homes of older adults using the Microsoft Kinect. IEEE J Biomed Health Inform, 19(1):290-301.

[31]WangJN, OliveiraMM, 2007. Filling holes on locally smooth surfaces reconstructed from point clouds. Image Vis Comput, 25(1):103-113.

[32]ZhengEL, WuCC, 2015. Structure from motion using structure-less resection. Proc IEEE Int Conf on Computer Vision, p.2075-2083.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE