Abstract: We propose a new framework for image-based three-dimensional (3D) model retrieval. We first model the query image as a Euclidean point. Then we model all projected views of a 3D model as a symmetric positive definite (SPD) matrix, which is a point on a Riemannian manifold. Thus, the image-based 3D model retrieval is reduced to a problem of Euclid-to-Riemann metric learning. To solve this heterogeneous matching problem, we map the euclidean space and SPD Riemannian manifold to the same high-dimensional hilbert space, thus shrinking the great gap between them. Finally, we design an optimization algorithm to learn a metric in this hilbert space using a kernel trick. Any new image descriptors, such as the features from deep learning, can be easily embedded in our framework. Experimental results show the advantages of our approach over the state-of-the-art methods for image-based 3D model retrieval.

利用流形学习进行基于图像的三维模型检索

[1]Bai S, Bai X, Zhou Z, et al., 2016. GIFT: a real-time and scalable 3D shape search engine. 16^th IEEE Conf on Computer Vision and Pattern Recognition, p.5023-5032.

[2]Bai X, Bai S, Zhu Z, et al., 2015. 3D shape matching via two layer coding. IEEE Trans Patt Anal Mach Intell, 37(12): 2361-2373.

[3]Cevikalp H, Triggs B, 2010. Face recognition based on image sets. IEEE Society Conf on Computer Vision and Pattern Recognition, p.2567-2573.

[4]Chatfield K, Simonyan K, Vedaldi A, et al., 2014. Return of the devil in the details: delving deep into convolutional nets. p.1-11. https://arxiv.org/abs/1405.3531

[5]Chen DY, Tian XP, Shen YT, et al., 2003. On visual similarity based 3D model retrieval. Comput Graph Forum, 22(3): 223-232.

[6]Chien JT, Wu CC, 2002. Discriminant waveletfaces and nearest feature classifiers for face recognition. IEEE Trans Patt Anal Mach Intell, 24(12):1644-1649.

[7]Eitz M, Richter R, Boubekeur T, et al., 2012. Sketch-based shape retrieval. ACM Trans Graph, 31(4):31-40.

[8]Furuya T, Ohbuchi R, 2013. Ranking on cross-domain manifold for sketch-based 3D model retrieval. Int Conf on Cyberworlds, p.274-281.

[9]Hamm J, Lee DD, 2008. Grassmann discriminant analysis: a unifying view on subspace-based learning. Proc 25^th Int Conf on Machine Learning, p.376-383.

[10]Hamm J, Lee DD, 2009. Extended Grassmann kernels for subspace-based learning. Advances in Neural Information Processing Systems, p.601-608.

[11]Huang Z, Wang R, Shan S, et al., 2014. Learning Euclidean- to-Riemannian metric for point-to-set classification. IEEE Conf on Computer Vision and Pattern Recognition, p.1677-1684.

[12]Jayasumana S, Hartley R, Salzmann M, et al., 2013. Kernel methods on the Riemannian manifold of symmetric positive definite matrices. IEEE Conf on Computer Vision and Pattern Recognition, p.73-80.

[13]Kazhdan M, Funkhouser T, Rusinkiewicz S, 2003. Rotation invariant spherical harmonic representation of 3D shape descriptors. Proc Eurographics/ACM SIGGRAPH Symp on Geometry Processing, p.156-164.

[14]Kim T, Kittler J, Cipolla R, 2007. Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans Patt Anal Mach Intell, 29(6): 1005-1018.

[15]Li B, Lu Y, Godil A, et al., 2014. A comparison of methods for sketch-based 3D shape retrieval. Comput Vis Image Underst, 119:57-80.

[16]Lian Z, Godil A, Sun X, et al., 2013. CM-BOF: visual similarity-based 3D shape retrieval using clock matching and bag-of-features. Mach Vis Appl, 24(8):1685-1704.

[17]Mu P, Zhang S, Ye X, 2017. A metric learning method for image-based 3D shape retrieval. Proc Int Conf on Data Mining, Communications and Information Technology, Article 17.

[18]Ohbuchi R, Osada K, Furuya T, et al., 2008. Salient local visual features for shape-based 3D model retrieval. IEEE Int Conf on Shape Modeling and Applications, p.93-102.

[19]Papadakis P, Pratikakis I, Theoharis T, et al., 2010. Panorama: a 3D shape descriptor based on panoramic views for unsupervised 3D object retrieval. Int J Comput Vis, 89(2-3):177-192.

[20]Saavedra JM, Bustos B, Schreck T, et al., 2012. Sketch-based 3D model retrieval using keyshapes for global and local representation. Proc 5^th Eurographics Conf on 3D Object Retrieval, p.47-50.

[21]Shilane P, Min P, Kazhdan M, et al., 2004. The Princeton Shape Benchmark. Proc Shape Modeling Applications, p.167-178.

[22]Sousa P, Fonseca MJ, 2010. Sketch-based retrieval of drawings using spatial proximity. J Vis Lang Comput, 21(2):69-80.

[23]Su H, Maji S, Kalogerakis E, et al., 2015. Multi-view convolutional neural networks for 3D shape recognition. IEEE Int Conf on Computer Vision, p.945-953.

[24]Tabia H, Laga H, Picard D, et al., 2014. Covariance descriptors for 3D shape matching and retrieval. IEEE Conf on Computer Vision and Pattern Recognition, p.4185-4192.

[25]Vemulapalli R, Pillai JK, Chellappa R, 2013. Kernel learning for extrinsic classification of manifold features. IEEE Conf on Computer Vision and Pattern Recognition, p.1782-1789.

[26]Vincent P, Bengio Y, 2001. K-local hyperplane and convex distance nearest neighbor algorithms. Proc 14^th Int Conf on Neural Information Processing Systems: Natural and Synthetic, p.985-992.

[27]Wang F, Kang L, Li Y, 2015. Sketch-based 3D shape retrieval using convolutional neural networks. IEEE Conf on Computer Vision and Pattern Recognition, p.1875-1883.

[28]Wang R, Guo H, Davis LS, et al., 2012. Covariance discriminative learning: a natural and efficient approach to image set classification. IEEE Conf on Computer Vision and Pattern Recognition, p.2496-2503.

[29]Wen Y, Zhang K, Li Z, et al., 2016. A discriminative feature learning approach for deep face recognition. European Conf on Computer Vision, p.499-515.

[30]Wu Z, Song S, Khosla A, et al., 2015. 3D shapenets: a deep representation for volumetric shapes. IEEE Conf on Computer Vision and Pattern Recognition, p.1912-1920.

[31]Yamaguchi O, Fukui K, Maeda K, 1998. Face recognition using temporal image sequence. Proc 3^rd IEEE Int Conf on Automatic Face and Gesture Recognition, p.318-323.

[32]Zhu P, Zhang L, Zuo W, et al., 2013. From point to set: extend the learning of distance metrics. IEEE Int Conf on Computer Vision, p.2664-2671.