Full Text:   <5307>

Summary:  <528>

CLC number: TN29;TP391.4

On-line Access: 2024-08-27

Received: 2023-10-17

Revision Accepted: 2024-05-08

Crosschecked: 2021-03-22

Cited: 0

Clicked: 7393

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Hao YU

https://orcid.org/0000-0001-9984-5051

Bo YUAN

https://orcid.org/0000-0002-3185-2690

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2022 Vol.23 No.4 P.653-660

http://doi.org/10.1631/FITEE.2000679


A three-dimensional measurement method for binocular endoscopes based on deep learning


Author(s):  Hao YU, Changjiang ZHOU, Wei ZHANG, Liqiang WANG, Qing YANG, Bo YUAN

Affiliation(s):  State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):   yuanbo@zju.edu.cn

Key Words:  Key words: Binocular endoscope, Three-dimensional measurement, Deep learning, Disparity estimation


Share this article to: More <<< Previous Article|

Hao YU, Changjiang ZHOU, Wei ZHANG, Liqiang WANG, Qing YANG, Bo YUAN. A three-dimensional measurement method for binocular endoscopes based on deep learning[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(4): 653-660.

@article{title="A three-dimensional measurement method for binocular endoscopes based on deep learning",
author="Hao YU, Changjiang ZHOU, Wei ZHANG, Liqiang WANG, Qing YANG, Bo YUAN",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="23",
number="4",
pages="653-660",
year="2022",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2000679"
}

%0 Journal Article
%T A three-dimensional measurement method for binocular endoscopes based on deep learning
%A Hao YU
%A Changjiang ZHOU
%A Wei ZHANG
%A Liqiang WANG
%A Qing YANG
%A Bo YUAN
%J Frontiers of Information Technology & Electronic Engineering
%V 23
%N 4
%P 653-660
%@ 2095-9184
%D 2022
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2000679

TY - JOUR
T1 - A three-dimensional measurement method for binocular endoscopes based on deep learning
A1 - Hao YU
A1 - Changjiang ZHOU
A1 - Wei ZHANG
A1 - Liqiang WANG
A1 - Qing YANG
A1 - Bo YUAN
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 23
IS - 4
SP - 653
EP - 660
%@ 2095-9184
Y1 - 2022
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2000679


Abstract: 
Abstract:In the practice of clinical endoscopy, the precise estimation of the lesion size is quite significant for diagnosis. In this paper, we propose a three-dimensional (3D) measurement method for binocular endoscopes based on deep learning, which can overcome the poor robustness of the traditional binocular matching algorithm in texture-less areas. A simulated binocular image dataset is created from the target 3D data obtained by a 3D scanner and the binocular camera is simulated by 3D rendering software to train a disparity estimation model for 3D measurement. The experimental results demonstrate that, compared with the traditional binocular matching algorithm, the proposed method improves the accuracy and disparity map generation speed by 48.9% and 90.5%, respectively. This can provide more accurate and reliable lesion size and improve the efficiency of endoscopic diagnosis.

基于深度学习的双目内窥镜三维测量方法

余浩1,周长江2,张伟1,王立强1,2,杨青1,2,袁波1
1浙江大学光电科学与工程学院现代光学仪器国家重点实验室,中国杭州市,310027
2之江实验室超级感知中心,中国杭州市,311100
摘要:在内窥镜临床检查中,病灶尺寸精确估计对诊断具有非常重要的意义。本文提出一种基于深度学习的双目内窥镜三维测量方法,可以克服传统双目匹配算法在弱纹理区域鲁棒性较差的缺点。利用三维扫描仪获得的目标三维数据和三维渲染软件仿真的双目相机创建虚拟双目图像数据集,用于训练视差预测模型进行三维测量。实验结果表明,所提方法相比传统双目匹配算法在视差准确度和视差图生成速度上分别提高48.9%和90.5%,能够提供更加准确、可靠的病灶尺寸信息,提高内窥镜诊断效率。

关键词:双目内窥镜;三维测量;深度学习;视差预测

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Ahmad I, Levine JB, Anderson JC, 2016. Endoscopic measurement of colorectal polyps: how do we measure up? Gastroenterology, 150(3):769-771.

[2]Anderson BW, Smyrk TC, Anderson KS, et al., 2016. Endoscopic overestimation of colorectal polyp size. Gastrointest Endosc, 83(1):201-208.

[3]Cai HF, Wang R, Li Y, et al., 2018. Role of 3D reconstruction in the evaluation of patients with lower segment oesophageal cancer. J Thorac Dis, 10(7):3940-3947.

[4]Dimas G, Bianchi F, Iakovidis DK, et al., 2020. Endoscopic single-image size measurements. Meas Sci Technol, 31(7):074010.

[5]Furukawa R, Aoyama M, Hiura S, et al., 2014. Calibration of a 3D endoscopic system based on active stereo method for shape measurement of biological tissues and specimen. Proc 36th Annual Int Conf of the IEEE Engineering in Medicine and Biology Society, p.4991-4994.

[6]Hirschmüller H, 2008. Stereo processing by semiglobal matching and mutual information. IEEE Trans Patt Anal Mach Intell, 30(2):328-341.

[7]Kendall A, Martirosyan H, Dasgupta S, et al., 2017. End-to-end learning of geometry and context for deep stereo regression. Proc IEEE Int Conf on Computer Vision, p.66-75.

[8]Khamis S, Fanello S, Rhemann C, et al., 2018. StereoNet: guided hierarchical refinement for real-time edge-aware depth prediction. https://arxiv.org/abs/1807.08865v1

[9]Nomura K, Kikuchi D, Kaise M, et al., 2019. Comparison of 3D endoscopy and conventional 2D endoscopy in gastric endoscopic submucosal dissection: an ex vivo animal study. Surg Endosc, 33(12):4164-4170.

[10]Ogino-Nishimura E, Nakagawa T, Sakamoto T, et al., 2015. Efficacy of three-dimensional endoscopy in endonasal surgery. Auris Nasus Larynx, 42(3):203-207.

[11]Omori J, Goto O, Higuchi K, et al., 2020. Three-dimensional flexible endoscopy can facilitate efficient and reliable endoscopic hand suturing: an ex-vivo study. Clin Endosc, 53(3):334-338.

[12]Sakata S, Mcivor F, Klein K, et al., 2018. Measurement of polyp size at colonoscopy: a proof-of-concept simulation study to address technology bias. Gut, 67(2):206-208.

[13]Shaw MJ, Shaukat A, 2016. Does polyp size scatter matter? Gastrointest Endosc, 83(1):209-211.

[14]Wang D, Liu H, Cheng X, 2018. A miniature binocular endoscope with local feature matching and stereo matching for 3D measurement and 3D reconstruction. Sensors, 18(7):2243.

[15]Wang XZ, Nie YF, Lu SP, et al., 2020. Deep convolutional network for stereo depth mapping in binocular endoscopy. IEEE Access, 8:73241-73249.

[16]Žbontar J, LeCun Y, 2014. Computing the stereo matching cost with a convolutional neural network. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1592-1599.

[17]Zhang FH, Prisacariu V, Yang RG, et al., 2019. GA-Net: guided aggregation net for end-to-end stereo matching. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.185-194.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE