CLC number: TP312
On-line Access: 2021-10-08
Received: 2020-10-20
Revision Accepted: 2021-03-26
Crosschecked: 2021-08-31
Cited: 0
Clicked: 5116
Citations: Bibtex RefMan EndNote GB/T7714
Xinya Wang, Jiayi Ma, Wenjing Gao, Junjun Jiang. MPIN: a macro-pixel integration network for light field super-resolution[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.2000566 @article{title="MPIN: a macro-pixel integration network for light field super-resolution", %0 Journal Article TY - JOUR
MPIN:基于宏像素聚合的光场图像超分辨率网络1武汉大学电子信息学院,中国武汉市,430072 2哈尔滨工业大学计算机科学与技术学院,中国哈尔滨市,150001 摘要:现有的大多数光场超分辨率方法不能充分利用角度信息,或者由于利用部分视图而产生不均衡的性能。为解决这些问题,本文提出一种基于宏像素表示的光场图像超分辨率聚合网络模型(称为MPIN)。该网络通过将四维光场图像重新排列成二维宏像素图像,将空间和角度信息进行耦合,从而同时恢复整张光场图像。网络利用两种特殊的卷积分别提取空间和角度信息。为充分利用空间-角度相关性,所设计的聚合残差模块融合两种信息使其相互引导,以实现角度相干性。在宏像素表示下,该网络通过扩展角度混洗层来提高宏像素图像的空间分辨率,有效避免了混叠。在合成和真实光场数据集上的大量实验表明,本文提出的方法在定性和定量上均实现了比现有方法更好的性能。此外,该方法在保持光场图像固有极线结构的同时,具有均衡性能分布的优点。 关键词组: Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article
Reference[1]Alain M, Smolic A, 2018. Light field super-resolution via LFBM5D sparse coding. Proc 25th IEEE Int Conf on Image Processing, p.2501-2505. [2]Bishop TE, Favaro P, 2012. The light field camera: extended depth of field, aliasing, and superresolution. IEEE Trans Patt Anal Mach Intell, 34(5):972-986. [3]Bishop TE, Zanetti S, Favaro P, 2009. Light field super-resolution. Proc IEEE Int Conf on Computational Photography, p.1-9. [4]Dong C, Loy CC, He KM, et al., 2014. Learning a deep convolutional network for image super-resolution. Proc 13th European Conf on Computer Vision, p.184-199. [5]Fan HZ, Liu D, Xiong ZW, et al., 2017. Two-stage convolutional neural network for light field super-resolution. Proc IEEE Int Conf on Image Processing, p.1167-1171. [6]Farrugia RA, Galea C, Guillemot C, 2017. Super resolution of light field images using linear subspace projection of patch-volumes. IEEE J Sel Top Signal Process, 11(7):1058-1071. [7]Glorot X, Bengio Y, 2010. Understanding the difficulty of training deep feedforward neural networks. Proc 13th Int Conf on Artificial Intelligence and Statistics, p.249-256. [8]Gul MSK, Gunturk BK, 2018. Spatial and angular resolution enhancement of light fields using convolutional neural networks. IEEE Trans Image Process, 27(5):2146-2159. [9]Honauer K, Johannsen O, Kondermann D, et al., 2016. A dataset and evaluation methodology for depth estimation on 4D light fields. Proc 13th Asian Conf on Computer Vision, p.19-34. [10]Jin J, Hou JH, Chen J, et al., 2020. Light field spatial super-resolution via deep combinatorial geometry embedding and structural consistency regularization. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.2257-2266. [11]Kalantari NK, Wang TC, Ramamoorthi R, 2016. Learning-based view synthesis for light field cameras. ACM Trans Graph, 35(6):193. [12]Levoy M, Hanrahan P, 1996. Light field rendering. Proc 23rd Annual Conf on Computer Graphics and Interactive Techniques, p.31-42. [13]Li M, Diao CY, Xu DQ, et al., 2020. A non-Lambertian photometric stereo under perspective projection. Front Inform Technol Electron Eng, 21(8):1191-1205. [14]Liang CK, Ramamoorthi R, 2015. A light transport framework for lenslet light field cameras. ACM Trans Graph, 34(2):16. [15]Lim B, Son S, Kim H, et al., 2017. Enhanced deep residual networks for single image super-resolution. Proc IEEE Conf on Computer Vision and Pattern Recognition Workshops, p.1132-1140. [16]Lim J, Ok H, Park B, et al., 2009. Improving the spatail resolution based on 4D light field data. Proc 16th IEEE Int Conf on Image Processing, p.1173-1176. [17]Mitra K, Veeraraghavan A, 2012. Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior. Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition Workshops, p.22-28. [18]Nava FP, Luke JP, 2009. Simultaneous estimation of super-resolved depth and all-in-focus images from a plenoptic camera. Proc 3DTV Conf: the True Vision—Capture, Transmission and Display of 3D Video, p.1-4. [19]Ng R, Levoy M, Brédif M, et al., 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Technical Report No. CTSR 2005-02, Stanford University, USA. [20]Peng JY, Xiong ZW, Liu D, et al., 2018. Unsupervised depth estimation from light field using a convolutional neural network. Proc Int Conf on 3D Vision, p.295-303. [21]Pérez F, Pérez A, Rodríguez M, et al., 2012. Fourier slice super-resolution in plenoptic cameras. Proc IEEE Int Conf on Computational Photography, p.1-11. [22]Pérez F, Pérez A, Rodríguez M, et al., 2015. Super-resolved Fourier-slice refocusing in plenoptic cameras. J Math Imag Vis, 52(2):200-217. [23]Rerabek M, Ebrahimi T, 2016. New light field image dataset. Proc 8th Int Conf on Quality of Multimedia Experience, p.1-2. [24]Rossi M, Frossard P, 2017. Graph-based light field super-resolution. Proc IEEE 19th Int Workshop on Multimedia Signal Processing, p.1-6. [25]Rossi M, Frossard P, 2018. Geometry-consistent light field super-resolution via graph-based regularization. IEEE Trans Image Process, 27(9):4207-4218. [26]Shi WZ, Caballero J, Huszár F, et al., 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1874-1883. [27]Wang TC, Zhu JY, Hiroaki E, et al., 2016. A 4D light-field dataset and CNN architectures for material recognition. Proc 14th European Conf on Computer Vision, p.121-138. [28]Wang YL, Hou GQ, Sun ZN, et al., 2016. A simple and robust super resolution method for light field images. Proc IEEE Int Conf on Image Processing, p.1459-1463. [29]Wang YL, Liu F, Zhang KB, et al., 2018. LFNet: a novel bidirectional recurrent convolutional neural network for light-field image super-resolution. IEEE Trans Image Process, 27(9):4274-4286. [30]Wang YQ, Wang LG, Yang JG, et al., 2020. Spatial-angular interaction for light field image super-resolution. Proc 16th European Conf on Computer Vision, p.290-308. [31]Wanner S, Goldluecke B, 2014. Variational light field analysis for disparity estimation and super-resolution. IEEE Trans Patt Anal Mach Intell, 36(3):606-619. [32]Wanner S, Meister S, Goldluecke B, 2013. Datasets and benchmarks for densely sampled 4D light fields. Proc 18th Int Workshop on Vision, Modeling, and Visualization, p.225-226. [33]Yeung HWF, Hou JH, Chen XM, et al., 2019. Light field spatial super-resolution using deep efficient spatial-angular separable convolution. IEEE Trans Image Process, 28(5):2319-2330. [34]Yi P, Wang ZY, Jiang K, et al., 2019. Progressive fusion video super-resolution network via exploiting non-local spatio-temporal correlations. Proc IEEE/CVF Int Conf on Computer Vision, p.3106-3115. [35]Yoon Y, Jeon HG, Yoo D, et al., 2015. Learning a deep convolutional network for light-field image super-resolution. Proc IEEE Int Conf on Computer Vision Workshop, p.57-65. [36]Yuan Y, Cao ZQ, Su LJ, 2018. Light-field image superresolution using a combined deep CNN based on EPI. IEEE Signal Process Lett, 25(9):1359-1363. [37]Yücer K, Sorkine-Hornung A, Wang O, et al., 2016. Efficient 3D object segmentation from densely sampled light fields with applications to 3D reconstruction. ACM Trans Graph, 35(3):22. [38]Zhang S, Lin YF, Sheng H, 2019. Residual networks for light field image super-resolution. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.11038-11047. [39]Zhu H, Wang Q, Yu JY, 2017. Light field imaging: models, calibrations, reconstructions, and applications. Front Inform Technol Electron Eng, 18(9):1236-1249. Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou
310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE |
Open peer comments: Debate/Discuss/Question/Opinion
<1>