Full Text:   <1199>

CLC number: TP391

On-line Access: 

Received: 2005-08-05

Revision Accepted: 2005-09-10

Crosschecked: 0000-00-00

Cited: 7

Clicked: 3227

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
1. Reference List
Open peer comments

Journal of Zhejiang University SCIENCE A 2005 Vol.6 No.11 P.1268~1283


Exploiting multi-context analysis in semantic image classification

Author(s):  TIAN Yong-hong, HUANG Tie-jun, GAO Wen

Affiliation(s):  Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100080, China; more

Corresponding email(s):   yhtian@ict.ac.cn, tjhuang@ict.ac.cn, wgao@ict.ac.cn

Key Words:  Image classification, Multi-context analysis, Cross-modal correlation analysis, Link-based correlation model, Linkage semantic kernels, Relational support vector classifier

TIAN Yong-hong, HUANG Tie-jun, GAO Wen. Exploiting multi-context analysis in semantic image classification[J]. Journal of Zhejiang University Science A, 2005, 6(11): 1268~1283.

@article{title="Exploiting multi-context analysis in semantic image classification",
author="TIAN Yong-hong, HUANG Tie-jun, GAO Wen",
journal="Journal of Zhejiang University Science A",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Exploiting multi-context analysis in semantic image classification
%A TIAN Yong-hong
%A HUANG Tie-jun
%A GAO Wen
%J Journal of Zhejiang University SCIENCE A
%V 6
%N 11
%P 1268~1283
%@ 1673-565X
%D 2005
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.2005.A1268

T1 - Exploiting multi-context analysis in semantic image classification
A1 - TIAN Yong-hong
A1 - HUANG Tie-jun
A1 - GAO Wen
J0 - Journal of Zhejiang University Science A
VL - 6
IS - 11
SP - 1268
EP - 1283
%@ 1673-565X
Y1 - 2005
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.2005.A1268

As the popularity of digital images is rapidly increasing on the Internet, research on technologies for semantic image classification has become an important research topic. However, the well-known content-based image classification methods do not overcome the so-called semantic gap problem in which low-level visual features cannot represent the high-level semantic content of images. image classification using visual and textual information often performs poorly since the extracted textual features are often too limited to accurately represent the images. In this paper, we propose a semantic image classification approach using multi-context analysis. For a given image, we model the relevant textual information as its multi-modal context, and regard the related images connected by hyperlinks as its link context. Two kinds of context analysis models, i.e., cross-modal correlation analysis and link-based correlation model, are used to capture the correlation among different modals of features and the topical dependency among images induced by the link structure. We propose a new collective classification model called relational support vector classifier (RSVC) based on the well-known Support Vector Machines (SVMs) and the link-based correlation model. Experiments showed that the proposed approach significantly improved classification accuracy over that of SVM classifiers using visual and/or textual features.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1] Cai, D., Yu, S., Wen, J.R., Ma, W.Y., 2003. VIPS: A Vision-base Page Segmentation Algorithm. Technical Report, MSR-TR-2003-79, Microsoft Research Asia.

[2] Cai, D., He, X.F., Li, Z.W., Ma, W.Y., Wen, J.R., 2004. Hierarchical Clustering of WWW Image Search Results Using Visual, Textual and Link Analysis. Proceedings of 12th ACM International Conference on Multimedia, New York, USA, p.952-959.

[3] Chen, Z., Liu, W.Y., Zhang, F., Li, M.J., Zhang, H.J., 2001. Web mining for Web image retrieval. Journal of American Society of Infomation Science and Technology, 52(10):831-839.

[4] Cristianini, N., Shawe-Talyor, J., Lodhi, H., 2002. Latent semantic kernels. Journal of Intelligent Information Systems, 18(2/3):127-152.

[5] Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R., 1990. Indexing by latent semantic analysis. Journal of American Society of Infomation Science and Technology, 41(6):389-401.

[6] Gärtner, T., 2003. A survey of kernels for structured data. SIGKDD Explorations, 5(1):49-58.

[7] Jensen, D., Neville, J., Gallagher, B., 2004. Why Collective Inference Improves Relational Classification. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA, p.593-598.

[8] Joachims, T., Cristianini, N., Shawe-Talyor, J., 2001. Composite Kernels for Hypertext Categorization. Proceedings of the International Conference on Machine Learning (ICML-2001), Morgan Kaunfmann Publishers, San Francisco, US. p.250-257.

[9] Kandola, J., Shawe-Talyor, J., Cristianini, N., 2002. Learning Semantic Similarity. Proceedings of International Conference on Advances in Information Processing System (NIPS).

[10] Kondor, R.I., Lafferty, J., 2002. Diffusion Kernels on Graphs and Other Discrete Structures. Proceedings of the 19th International Conference on Machine Learning (ICML02), p.315-322.

[11] Lempel, R., Soffer, A., 2002. PicASHOW: pictorial authority search by hyperlinks on the Web. ACM Transactions on Information Systems, 20(1):1-24.

[12] Li, D.G., Dimitrova, N., Li, M.K., Sethi, I.K., 2003. Multimedia Content Processing through Cross-modal Association. Proceedings of 11th ACM International Conference on Multimedia, Berkeley, California, USA, p.604-611.

[13] Ma, W.Y., Zhang, H.J., 1998. Content-based Image Indexing and Retrieval. In: Furht, B. (Ed.), Handbook of Multimedia Computing. CRC Press, Boca Raton, FL.

[14] Mohr, J., Obermayer, K., 2005. A Topographic Support Vector Machine: Classification Using Local Label Configurations. Advances in Neural Information Processing Systems 17, MIT Press, Cambridge, MA, p.929-936.

[15] Neville, J., Jensen, D., 2003. Collective Classification with Relational Dependency Networks. Proceedings of 2nd Multi-Relational Data Mining Workshop, 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, p.77-91.

[16] Paek, S., Sable, C.L., Hatzivassiloglou, V., Jaimes, A., Schiffman, B.H., Chang, S.F., McKeown, K.R., 1999. Integration of Visual and Text-Based Approaches for the Content Labeling and Classification of Photographs. Proceedings of ACM SIGIR’99, Workshop on Multimedia Indexing and Retrieval.

[17] Schölkopf, B., 2000. Statistical Learning and Kernel Methods. Technical Report, MSR-TR-2000-23, Microsoft Research.

[18] Siolas, G., d’Alché-Buc, F., 2000. Support Machine Learning Based on Semantic Kernel for Text Categorization. Proceedings of the International Joint Conference on Neural Network (IJCNN).

[19] Tian, Y.H., Huang, T.J., Gao, W., 2004. Two-phase Web site classification based on Hidden Markov Tree models. International Journal: Web Intelligence and Agent System, 4(2):249-264.

[20] Tian, Y.H., Huang, T.J., Gao, W., 2005. Latent linkage semantic kernels for collective classification of link data. Journal of Intelligent Information Systems (in Press).

[21] Vailaya, A., Figueiredo, M., A.T., Jain, A.K., Zhang, H.J., 2001. Image classification for content-based indexing. IEEE Trans. Image Processing, 10(1):117-129.

[22] Wang, X.J., Ma, W.Y., Xue, G.R., Li, X., 2004. Multi-Model Similarity Propagation and its Application for Web Image Retrieval. Proceedings of 12th ACM International Conference on Multimedia, New York, USA, p.944-951.

[23] Yang, Y., Slattery, S., Ghani, R., 2002. A study of approaches to hypertext categorization. Journal of Intelligent Information System, 18(2/3):219-241.

[24] Yu, H., Li, M., Zhang, H.J., Feng, J., 2002. Color Exture Moments for Content-based Image Retrieval. International Conference on Image Processing, p.28.

[25] Zhao, R., Grosky, W.I., 2002. Narrowing the emantic gap—improved text-based Web document retrieval using visual features. IEEE Trans. Multimedia, 4(2):189-200.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE