JZUS - Journal of Zhejiang University SCIENCE

Frontiers of Information Technology & Electronic Engineering

Accepted manuscript available online (unedited version)

Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning

Author(s): Wanghui DING, Kaiwei SUN, Mengfei YU, Hangzheng LIN, Yang FENG, Jianhua LI, Zuozhu LIU
Affiliation(s): Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine, Zhejiang Provincial Clinical Research Center for Oral Diseases, Key Laboratory of Oral Biomedical Research of Zhejiang Province, Cancer Center of Zhejiang University, Engineering Research Center of Oral Biomaterials and Devices of Zhejiang Province, Hangzhou 310000, China; more
Corresponding email(s): godson888@zju.edu.cn, zuozhuliu@intl.zju.edu.cn
Key Words: Artificial intelligence; Digital dentistry; Deep learning; Orthodontics; Tooth pose; Neural network

Share this article to： More <<< Previous Paper \|Next Paper >>>

Wanghui DING, Kaiwei SUN, Mengfei YU, Hangzheng LIN, Yang FENG, Jianhua LI, Zuozhu LIU. Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.2300596

@article{title="Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning",
author="Wanghui DING, Kaiwei SUN, Mengfei YU, Hangzheng LIN, Yang FENG, Jianhua LI, Zuozhu LIU",
journal="Frontiers of Information Technology & Electronic Engineering",
year="in press",
publisher="Zhejiang University Press & Springer",
doi="https://doi.org/10.1631/FITEE.2300596"
}

%0 Journal Article
%T Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning
%A Wanghui DING
%A Kaiwei SUN
%A Mengfei YU
%A Hangzheng LIN
%A Yang FENG
%A Jianhua LI
%A Zuozhu LIU
%J Frontiers of Information Technology & Electronic Engineering
%P 1240-1249
%@ 2095-9184
%D in press
%I Zhejiang University Press & Springer
doi="https://doi.org/10.1631/FITEE.2300596"

TY - JOUR
T1 - Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning
A1 - Wanghui DING
A1 - Kaiwei SUN
A1 - Mengfei YU
A1 - Hangzheng LIN
A1 - Yang FENG
A1 - Jianhua LI
A1 - Zuozhu LIU
J0 - Frontiers of Information Technology & Electronic Engineering
SP - 1240
EP - 1249
%@ 2095-9184
Y1 - in press
PB - Zhejiang University Press & Springer
ER -
doi="https://doi.org/10.1631/FITEE.2300596"

Abstract
Chinese Summary
Academic Network
Reviewer Comment

Abstract: A critical step in digital dentistry is to accurately and automatically characterize the orientation and position of individual teeth, which can subsequently be used for treatment planning and simulation in orthodontic tooth alignment. This problem remains challenging because the geometric features of different teeth are complicated and vary significantly, while a reliable large-scale dataset is yet to be constructed. In this paper we propose a novel method for automatic tooth orientation estimation by formulating it as a six-degree-of-freedom (6-DoF) tooth pose estimation task. Regarding each tooth as a three-dimensional (3D) point cloud, we design a deep neural network with a feature extractor backbone and a two-branch estimation head for tooth pose estimation. Our model, trained with a novel loss function on the newly collected large-scale dataset (10 393 patients with 280 611 intraoral tooth scans), achieves an average Euler angle error of only 4.780°–5.979° and a translation L1 error of 0.663 mm on a hold-out set of 2598 patients (77 870 teeth). Comprehensive experiments show that 98.29% of the estimations produce a mean angle error of less than 15°, which is acceptable for many clinical and industrial applications.

基于深度学习的口腔三维扫描中六方位自由度牙齿姿态准确估算

丁王辉¹，孙凯伟²，俞梦飞¹，林航正²，冯洋³，李建华⁴，刘佐珠^1,2
¹浙江大学医学院附属口腔医院·浙江大学口腔医学院·浙江省口腔疾病临床医学研究中心·浙江省口腔生物医学研究重点实验室·浙江大学癌症研究院·口腔生物材料与器械浙江省工程研究中心，中国杭州市，310000
²浙江大学伊利诺伊大学厄巴纳香槟校区联合学院，中国海宁市，314400
³上海时代天使医疗器械有限公司，中国上海市，200433
⁴杭州口腔医院，中国杭州市，310006
摘要：数字牙科的一个关键步骤是准确、自动地表征牙齿的方向和位置，在此基础上可以辅助制定正畸治疗计划和模拟牙齿排齐。由于不同牙齿之间的几何特征复杂且差异较大，且可靠的大规模数据集尚未构建，表征牙齿的方向和位置仍然具有挑战性。本文提出一种新的牙齿方位自动估算方法，将其表述为六方位自由度的牙齿姿态估算任务。将每个牙齿视为一个三维点云，设计了一个具有特征提取主干和双支路检测头的深度神经网络模型，以估算牙齿姿态。使用新的损失函数训练新收集的大样本数据集（10 393例患者，280 611颗牙齿的扫描数据），在2598例患者（77 870颗牙齿）的数据集上，平均欧拉角误差仅为4.780°–5.979°，平移L1误差为0.663 mm。综合实验表明，98.29%的估算产生的平均角度误差小于15°，这对于大多数临床和工业应用可以接受。

关键词组：人工智能；数字牙科；深度学习；口腔正畸；牙齿姿态；神经网络

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Cai M, Reid I, 2020. Reconstruct locally, localize globally: a model free method for object pose estimation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p. 3150-3160.

[2]Chen QM, Wang YH, Shuai J, 2023. Current status and future prospects of stomatology research. J Zhejiang Univ-Sci B (Biomed & Biotechnol), 24(10):853-867.

[3]Gu CH, Ren XF, 2010. Discriminative mixture-of-templates for viewpoint classification. 11^th European Conf on Computer Vision, p.408-421.

[4]Herrmann W, 1967. On the completion of Fédération Dentaire Internationale Specifications. Zahn Mitteil, 57(23):‍1147-1149 (in German).

[5]Hinterstoisser S, Cagniart C, Ilic S, et al., 2012. Gradient response maps for real-time detection of textureless objects. IEEE Trans Patt Anal Mach Intell, 34(5):876-888.

[6]Kendall A, Grimes M, Cipolla R, 2015. PoseNet: a convolutional network for real-time 6-DOF camera relocalization. Proc IEEE Int Conf on Computer Vision, p.2938-2946.

[7]Li ZG, Wang G, Ji XY, 2019. CDPN: coordinates-based disentangled pose network for real-time RGB-based 6-DoF object pose estimation. Proc IEEE/CVF Int Conf on Computer Vision, p.7677-7686.

[8]Liebelt J, Schmid C, Schertler K, 2008. Viewpoint-independent object class detection using 3D feature maps. IEEE Conf on Computer Vision and Pattern Recognition, p.1-8.

[9]Mok V, Ong SH, Foong KWC, et al., 2002. Pose estimation of teeth through crown-shape matching. Proc SPIE 4684, Medical Imaging 2002: Image Processing, p.955-964.

[10]Newell A, Yang KY, Deng J, 2016. Stacked hourglass networks for human pose estimation. 14^th European Conf on Computer Vision, p.483-499.

[11]Oberweger M, Rad M, Lepetit V, 2018. Making deep heatmaps robust to partial occlusions for 3D object pose estimation. Proc 15^th European Conf on Computer Vision, p.125-141.

[12]Park K, Patten T, Vincze M, 2019. Pix2Pose: pixel-wise coordinate regression of objects for 6D pose estimation. Proc IEEE/CVF Int Conf on Computer Vision, p.7667-7676.

[13]Peng SD, Liu Y, Huang QX, et al., 2019. PVNet: pixel-wise voting network for 6DoF pose estimation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.4556-4565.

[14]Qi CR, Litany O, He KM, et al., 2019. Deep Hough voting for 3D object detection in point clouds. Proc IEEE/CVF Int Conf on Computer Vision, p.9276-9285.

[15]Su H, Qi CR, Li YY, et al., 2015. Render for CNN: viewpoint estimation in images using CNNs trained with rendered 3D model views. Proc IEEE Int Conf on Computer Vision, p.2686-2694.

[16]Sun M, Bradski G, Xu BX, et al., 2010. Depth-encoded Hough voting for joint object detection and shape recovery. 11^th European Conf on Computer Vision, p.658-671.

[17]Ulrich J, Alsayed A, Arvin F, et al., 2022. Towards fast fiducial marker with full 6 DOF pose estimation. Proc 37^th ACM/SIGAPP Symp on Applied Computing, p.723-730.

[18]Wang C, Xu DF, Zhu YK, et al., 2019. DenseFusion: 6D object pose estimation by iterative dense fusion. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3338-3347.

[19]Wang H, Sridhar S, Huang JW, et al., 2019. Normalized object coordinate space for category-level 6D object pose and size estimation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.2637-2646.

[20]Wang Y, Sun YB, Liu ZW, et al., 2019. Dynamic graph CNN for learning on point clouds. ACM Trans Graph, 38(5):146.

[21]Wei GD, Cui MZ, Liu YM, et al., 2020. TANet: towards fully automatic tooth arrangement. 16^th European Conf on Computer Vision, p.481-497.

[22]Xiang Y, Schmidt T, Narayanan V, et al., 2018. PoseCNN: a convolutional neural network for 6D object pose estimation in cluttered scenes. Proc 14th Robotics: Science and Systems.

[23]Zhou Y, Tuzel O, 2018. VoxelNet: end-to-end learning for point cloud based 3D object detection. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.4490-4499.

[24]Zhou Y, Barnes C, Lu JW, et al., 2019. On the continuity of rotation representations in neural networks. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.5738-5746.

[25]Zhu JJ, Yang YX, Wong HM, 2023. Development and accuracy of artificial intelligence-generated prediction of facial changes in orthodontic treatment: a scoping review. J Zhejiang Univ-Sci B (Biomed & Biotechnol), 24(11):974-984.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

- Go to

基于深度学习的口腔三维扫描中六方位自由度牙齿姿态准确估算

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference