CLC number: TP391.4
On-line Access: 2022-07-21
Received: 2021-04-07
Revision Accepted: 2021-11-10
Crosschecked: 2022-07-21
Cited: 0
Clicked: 2787
Citations: Bibtex RefMan EndNote GB/T7714
Chen JIA, Fan SHI, Meng ZHAO, Shengyong CHEN. Light field imaging for computer vision: a survey[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.2100180 @article{title="Light field imaging for computer vision: a survey", %0 Journal Article TY - JOUR
用于计算机视觉任务的光场成像技术综述1天津理工大学学习型智能系统教育部工程研究中心,中国天津市,300384 2天津理工大学计算机视觉与系统教育部重点实验室,中国天津市,300384 摘要:光场成像因其解决计算机视觉问题的能力而备受关注。本文首先简要回顾了近年来计算机视觉的研究进展。对于影响计算机视觉发展的大多数因素来说,视觉信息获取的丰富性和准确性起着决定性作用。光场成像技术利用照相机或微透镜阵列记录光线位置和方向信息,获取完整三维场景信息,为计算机视觉研究做出巨大贡献。光场成像提高了深度估计以及图像分割、融合和三维重建的精度。光场成像还被创新地应用于虹膜和人脸识别、材料和虚假行人识别、极平面图像采集和形状恢复以及光场显微镜。我们进一步总结了光场成像技术在计算机视觉研究中存在的问题和发展趋势,如光场数据集的建立和评估、在高动态范围条件下的应用、光场增强和虚拟现实。光场成像在各种研究中取得巨大成功。在过去25年,超过180篇文献报道了光场成像在解决计算机视觉问题上的能力。我们梳理了这些文献,使研究人员更容易搜索有关解决方案的详细方法。 关键词组: Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article
Reference[1]Adelson EH, Bergen JR, 1991. The plenoptic function and the elements of early vision. In: Landy MS, Movshon JA (Eds.), Computational Models of Visual Processing. MIT Press, Cambridge, USA, p.3-20. [2]Afshari H, Akin A, Popovic V, et al., 2012. Real-time FPGA implementation of linear blending vision reconstruction algorithm using a spherical light field camera. Proc IEEE Workshop on Signal Processing Systems, p.49-54. [3]Alperovich A, Goldluecke B, 2017. A variational model for intrinsic light field decomposition. Proc 13th Asian Conf on Computer Vision, p.66-82. [4]Balogh T, Kovács PT, 2010. Real-time 3D light field transmission. Proc SPIE 7724, Real-Time Image and Video Processing, Article 772406. [5]Berent J, Dragotti PL, 2007. Segmentation of epipolar-plane image volumes with occlusion and disocclusion competition. Proc IEEE Workshop on Multimedia Signal Processing, p.182-185. [6]Broxton M, Grosenick L, Yang S, et al., 2013. Wave optics theory and 3-D deconvolution for the light field microscope. Opt Expr, 21(21):25418-25439. [7]Campbell NDF, Vogiatzis G, Hernández C, et al., 2010. Automatic 3D object segmentation in multiple views using volumetric graph-cuts. Image Vis Comput, 28(1):14-25. [8]Campbell NDF, Vogiatzis G, Hernandez C, et al., 2011. Automatic object segmentation from calibrated images. Proc Conf for Visual Media Production, p.126-137. [9]Chen XY, Dai F, Ma YK, et al., 2015. Automatic foreground segmentation using light field images. Proc Visual Communications and Image Processing, p.1-4. [10]Cheng Z, Xiong ZW, Chen C, et al., 2019. Light field super-resolution: a benchmark. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops, p.1804-1813. [11]Cohen N, Yang S, Andalman A, et al., 2014. Enhancing the performance of the light field microscope using wavefront coding. Opt Expr, 22(20):24817-24839. [12]Criminisi A, Kang SB, Swaminathan R, et al., 2005. Extracting layers and analyzing their specular properties using epipolar-plane-image analysis. Comput Vis Image Underst, 97(1):51-85. [13]Cui YL, Yu M, Jiang ZD, et al., 2021. Blind light field image quality assessment by analyzing angular-spatial characteristics. Dig Signal Process, 117:103138. [14]Fang YM, Wei KK, Hou JH, et al., 2018. Light filed image quality assessment by local and global features of epipolar plane image. Proc IEEE 4th Int Conf on Multimedia Big Data, p.1-6. [15]Fiss J, Curless B, Szeliski R, 2014. Refocusing plenoptic images using depth-adaptive splatting. Proc IEEE Int Conf on Computational Photography, p.1-9. [16]Gao Q, Han L, Shen J, et al., 2017. Focused-region segmentation for light field images based on PCNN. Proc Int Smart Cities Conf, p.1-6. [17]Georgiev TG, Lumsdaine A, 2010. Focused plenoptic camera and rendering. J Electron Imag, 19(2):021106. [18]Gershun A, 1939. The light field. J Math Phys, 18(1-4):51-151. [19]Ghasemi A, Vetterli M, 2014. Detecting planar surface using a light-field camera with application to distinguishing real scenes from printed photos. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.4588-4592. [20]Gryaditskaya Y, Masia B, Didyk P, et al., 2016. Gloss editing in light fields. Proc Conf on Vision, Modeling and Visualization, p.127-135. [21]Guo BC, Wen JT, Han YX, 2020. Deep material recognition in light-fields via disentanglement of spatial and angular information. Proc 16th European Conf on Computer Vision, p.664-679. [22]Guo XQ, Lin HT, Yu Z, et al., 2015. Barcode imaging using a light field camera. Proc European Conf on Computer Vision, p.519-532. [23]Hog M, Sabater N, Guillemot C, 2016. Light field segmentation using a ray-based graph structure. Proc 14th European Conf on Computer Vision, p.35-50. [24]Hsieh PY, Chou PY, Lin HA, et al., 2018. Long working range light field microscope with fast scanning multifocal liquid crystal microlens array. Opt Expr, 26(8):10981-10996. [25]Huang ZJ, Yu M, Xu HY, et al., 2018. New quality assessment method for dense light fields. Proc SPIE 10817, Optoelectronic Imaging and Multimedia Technology V, Article 1081717. [26]Jia C, Shi F, Zhao YF, et al., 2018. Identification of pedestrians from confused planar objects using light field imaging. IEEE Access, 6:39375-39384. [27]Johannsen O, Sulc A, Goldluecke B, 2015. Variational separation of light field layers. Proc 20th Int Symp on Vision, Modeling, and Visualization, p.135-142. [28]Kalantari NK, Wang TC, Ramamoorthi R, 2016. Learning-based view synthesis for light field cameras. ACM Trans Graph, 35(6):193. [29]Kim C, Zimmer H, Pritch Y, et al., 2013. Scene reconstruction from high spatio-angular resolution light fields. ACM Trans Graph, 32(4):73. [30]Lee JY, Park RH, 2017. Separation of foreground and background from light field using gradient information. Appl Opt, 56(4):1069-1078. [31]Levoy M, Hanrahan P, 1996. Light field rendering. Proc 23rd Annual Conf on Computer Graphics and Interactive Techniques, p.31-42. [32]Levoy M, Ng R, Adams A, et al., 2006. Light field microscopy. Proc ACM SIGGRAPH, p.924-934. [33]Li NY, Ye JW, Ji Y, et al., 2014. Saliency detection on light field. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2806-2813. [34]Li NY, Sun BL, Yu JY, 2015. A weighted sparse coding framework for saliency detection. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.5216-5223. [35]Li ZQ, Xu ZX, Ramamoorthi R, et al., 2017. Robust energy minimization for BRDF-invariant shape from light fields. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.578-586. [36]Liang CK, Lin TH, Wong BY, et al., 2008. Programmable aperture photography: multiplexed light field acquisition. ACM Trans Graph, 27(3):1-10. [37]Lippmann G, 1908. Epreuves reversibles, photographies integrales. Comput R Acad Sci, 444:446-451. [38]Lumsdaine A, Georgiev T, 2009. The focused plenoptic camera. Proc IEEE Int Conf on Computational Photography, p.1-8. [39]Lv XQ, Wang X, Wang Q, et al., 2021. 4D light field segmentation from light field super-pixel hypergraph representation. IEEE Trans Vis Comput Graph, 27(9):3597-3610. [40]Marquez M, Rueda-Chacon H, Arguello H, 2020. Compressive spectral light field image reconstruction via online tensor representation. IEEE Trans Image Process, 29:3558-3568. [41]Mehajabin N, Pourazad M, Nasiopoulos P, 2020. SSIM assisted pseudo-sequence-based prediction structure for light field video compression. Proc IEEE Int Conf on Consumer Electronics, p.1-2. [42]Meng CL, An P, Huang XP, et al., 2019. Objective quality assessment for light field based on refocus characteristic. Proc 10th Int Conf on Image and Graphics, p.193-204. [43]Mihara H, Funatomi T, Tanaka K, et al., 2016. 4D light field segmentation with spatial and angular consistencies. Proc IEEE Int Conf on Computational Photography, p.1-8. [44]Murgia F, Giusto D, Perra C, et al., 2015. 3D reconstruction from plenoptic image. Proc 23rd Telecommunications Forum Telfor, p.448-451. [45]Ng R, Levoy M, Brédif M, et al., 2005. Light field photography with a hand-held plenoptic camera. Stanford Tech Report CTSR 2005-02. [46]Nian ZC, Jung C, 2019. CNN-based multi-focus image fusion with light field data. Proc IEEE Int Conf on Image Processing, p.1044-1048. [47]Paudyal P, Olsson R, Sjöström M, et al., 2016. SMART: a light field image quality dataset. Proc 7th Int Conf on Multimedia Systems, Article 49. [48]Paudyal P, Battisti F, Sjöström M, et al., 2017. Towards the perceptual quality evaluation of compressed light field images. IEEE Trans Broadcast, 63(3):507-522. [49]Paudyal P, Battisti F, Carli M, 2019. Reduced reference quality assessment of light field images. IEEE Trans Broadcast, 65(1):152-165. [50]Piao YR, Li X, Zhang M, et al., 2019a. Saliency detection via depth-induced cellular automata on light field. IEEE Trans Image Process, 29:1879-1889. [51]Piao YR, Rong ZK, Zhang M, et al., 2019b. Deep light-field-driven saliency detection from a single view. Proc 28th Int Joint Conf on Artificial Intelligence, p.904-911. [52]Piao YR, Jiang YY, Zhang M, et al., 2021. PANet: patch-aware network for light field salient object detection. IEEE Trans Cybern, early access. [53]Raghavendra R, Busch C, 2014. Presentation attack detection on visible spectrum iris recognition by exploring inherent characteristics of light field camera. Proc IEEE Int Joint Conf on Biometrics, p.1-8. [54]Raghavendra R, Raja KB, Yang B, et al., 2013a. Combining iris and periocular recognition using light field camera. Proc 2nd IAPR Asian Conf on Pattern Recognition, p.155-159. [55]Raghavendra R, Raja KB, Yang B, et al., 2013b. A novel image fusion scheme for robust multiple face recognition with light-field camera. Proc 16th Int Conf on Information Fusion, p.722-729. [56]Raghavendra R, Raja KB, Busch C, 2016. Exploring the usefulness of light field cameras for biometrics: an empirical study on face and iris recognition. IEEE Trans Inform Forens Secur, 11(5):922-936. [57]Rerabek M, Ebrahimi T, 2016. New light field image dataset. Proc 8th Int Conf on Quality of Multimedia Experience. [58]Sabater N, Boisson G, Vandame B, et al., 2017. Dataset and pipeline for multi-view light-field video. Proc IEEE Conf on Computer Vision and Pattern Recognition Workshops, p.1743-1753. [59]Sepas-Moghaddam A, Pereira F, Correia PL, 2018. Light field-based face presentation attack detection: reviewing, benchmarking and one step further. IEEE Trans Inform Forens Secur, 13(7):1696-1709. [60]Shan L, An P, Meng CL, et al., 2019. A no-reference image quality assessment metric by multiple characteristics of light field images. IEEE Access, 7:127217-127229. [61]Sheng H, Deng SY, Zhang S, et al., 2016. Segmentation of light field image with the structure tensor. Proc IEEE Int Conf on Image Processing, p.1469-1473. [62]Shi LK, Zhao SY, Zhou W, et al., 2018. Perceptual evaluation of light field image. Proc 25th IEEE Int Conf on Image Processing, p.41-45. [63]Shi LK, Zhao SY, Chen ZB, 2019. Belif: blind quality evaluator of light field image with tensor structure variation index. Proc IEEE Int Conf on Image Processing, p.3781-3785. [64]Shi LK, Zhou W, Chen ZB, et al., 2020. No-reference light field image quality assessment based on spatial-angular measurement. IEEE Trans Circ Syst Video Technol, 30(11):4114-4128. [65]Smith BM, Zhang L, Jin HL, et al., 2009. Light field video stabilization. Proc IEEE 12th Int Conf on Computer Vision, p.341-348. [66]Sulc A, Alperovich A, Marniok N, et al., 2016. Reflection separation in light fields based on sparse coding and specular flow. Proc Conf on Vision, Modeling and Visualization, p.137-144. [67]Sun J, Hossain M, Xu CL, et al., 2017. A novel calibration method of focused light field camera for 3-D reconstruction of flame temperature. Opt Commun, 390:7-15. [68]Tambe S, Veeraraghavan A, Agrawal A, 2013. Towards motion aware light field video for dynamic scenes. Proc IEEE Int Conf on Computer Vision, p.1009-1016. [69]Tao MW, Srinivasan PP, Malik J, et al., 2015a. Depth from shading, defocus, and correspondence using light-field angular coherence. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1940-1948. [70]Tao MW, Su JC, Wang TC, et al., 2015b. Depth estimation and specular removal for glossy surfaces using point and line consistency with light-field cameras. IEEE Trans Patt Anal Mach Intell, 38(6):1155-1169. [71]Tian Y, Zeng HQ, Hou JH, et al., 2021. A light field image quality assessment model based on symmetry and depth features. IEEE Trans Circ Syst Video Technol, 31(5):2046-2050. [72]Vizcaíno JP, Saltarin F, Belyaev Y, et al., 2021. Learning to reconstruct confocal microscopy stacks from single light field images. IEEE Trans Comput Imag, 7:775-788. [73]Wang AZ, Wang MH, Li XY, et al., 2017. A two-stage Bayesian integration framework for salient object detection on light field. Neur Process Lett, 46(3):1083-1094. [74]Wang HQ, Xu CX, Wang XZ, et al., 2016. Light field imaging based accurate image specular highlight removal. PLoS ONE, 11(6):e0156173. [75]Wang TC, Efros AA, Ramamoorthi R, 2015. Occlusion-aware depth estimation using light-field cameras. Proc IEEE Int Conf on Computer Vision, p.3487-3495. [76]Wang TC, Zhu JY, Hiroaki E, et al., 2016a. A 4D light-field dataset and CNN architectures for material recognition. Proc 14th European Conf on Computer Vision, p.121-138. [77]Wang TC, Chandraker M, Efros AA, et al., 2016b. SVBRDF-invariant shape and reflectance estimation from light-field cameras. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.5451-5459. [78]Wang TC, Zhu JY, Kalantari NK, et al., 2017. Light field video capture using a learning-based hybrid imaging system. ACM Trans Graph, 36(4):133. [79]Wang TT, Piao YR, Li XC, et al., 2019. Deep learning for light field saliency detection. Proc IEEE/CVF Int Conf on Computer Vision, p.8837-8847. [80]Wang YQ, Yang JG, Xiao C, et al., 2018. An efficient method for the fusion of light field refocused images. Proc SPIE 9th Int Conf on Graphic and Image Processing, Article 1061536. [81]Wanner S, Meister S, Goldluecke B, 2013a. Datasets and benchmarks for densely sampled 4D light fields. Proc 18th Int Workshop on Vision, Modeling, and Visualization, p.225-226. [82]Wanner S, Straehle C, Goldluecke B, 2013b. Globally consistent multi-label assignment on the ray space of 4D light fields. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1011-1018. [83]Wilburn B, Smulski M, Lee HHK, et al., 2002. Light field video camera. Proc SPIE 6474, Media Processors, p.29-36. [84]Wilburn B, Joshi N, Vaish V, et al., 2005. High performance imaging using large camera arrays. ACM Trans Graph, 24(3):765-776. [85]Wu GC, Masia B, Jarabo A, et al., 2017. Light field image processing: an overview. IEEE J Sel Top Signal Process, 11(7):926-954. [86]Xu YC, Nagahara H, Shimada A, et al., 2015. TransCut: transparent object segmentation from a light-field image. Proc IEEE Int Conf on Computer Vision, p.3442-3450. [87]Xu YC, Nagahara H, Shimada A, et al., 2019. TransCut2: transparent object segmentation from a light-field image. IEEE Trans Comput Imag, 5(3):465-477. [88]Yang JC, 2000. A Light Field Camera for Image Based Rendering. MS Thesis, Massachusetts Institute of Technology, Cambridge, USA. [89]Yücer K, Sorkine-Hornung A, Wang O, et al., 2016. Efficient 3D object segmentation from densely sampled light fields with applications to 3D reconstruction. ACM Trans Graph, 35(3):22. [90]Zhang C, Chen T, 2004. A self-reconfigurable camera array. Proc ACM SIGGRAPH Sketches, p.151. [91]Zhang C, Hou GQ, Sun ZA, et al., 2013. Light field photography for iris image acquisition. Proc 8th Chinese Conf on Biometric Recognition, p.345-352. [92]Zhang J, Wang M, Gao J, et al., 2015. Saliency detection with a deeper investigation of light field. Proc 24th Int Joint Conf on Artificial Intelligence, p.2212-2218. [93]Zhang J, Wang M, Lin L, et al., 2017. Saliency detection on light field: a multi-cue approach. ACM Trans Multim Comput Commun Appl, 13(3):32. [94]Zhang J, Liu YM, Zhang SP, et al., 2020. Light field saliency detection with deep convolutional networks. IEEE Trans Image Process, 29:4421-4434. [95]Zhang M, Geng Z, Pei RJ, et al., 2017. Three-dimensional light field microscope based on a lenslet array. Opt Commun, 403:133-142. [96]Zhang M, Li JJ, Wei J, et al., 2019. Memory-oriented decoder for light field salient object detection. Proc Advances in Neural Information Processing Systems32, p.2898-2909. [97]Zhang XD, Wang Y, Zhang J, et al., 2015. Light field saliency vs. 2D saliency: a comparative study. Neurocomputing, 166:389-396. [98]Zhou MY, Ding YQ, J Yiet al., 2020. Shape and reflectance reconstruction using concentric multi-spectral light field. IEEE Trans Patt Anal Mach Intell, 42(7):1594-1605. [99]Zhou W, Shi LK, Chen ZB, et al., 2020. Tensor oriented no-reference light field image quality assessment. IEEE Trans Image Process, 29:4070-4084. [100]Zhu H, Zhang Q, Wang Q, 2017. 4D light field superpixel and segmentation. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.6709-6717. Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou
310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE |
Open peer comments: Debate/Discuss/Question/Opinion
<1>