Full Text:   <3200>

CLC number: TN919.8

On-line Access: 

Received: 2008-06-21

Revision Accepted: 2008-10-28

Crosschecked: 2009-03-31

Cited: 2

Clicked: 5754

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
1. Reference List
Open peer comments

Journal of Zhejiang University SCIENCE A 2009 Vol.10 No.6 P.767-777


Dominant edge direction based fast intra mode decision in the H.264/AVC encoder

Author(s):  Byeongdu LA, Minyoung EOM, Yoonsik CHOE

Affiliation(s):  Department of Electrical and Electronic Engineering, Yonsei University, Seoul 120-749, Korea

Corresponding email(s):   bdla6077@yonsei.ac.kr, emy0606@yonsei.ac.kr, yschoe@yonsei.ac.kr

Key Words:  H.264, Intra prediction, Mode decision, Dominant edge direction

Share this article to: More |Next Article >>>

Byeongdu LA, Minyoung EOM, Yoonsik CHOE. Dominant edge direction based fast intra mode decision in the H.264/AVC encoder[J]. Journal of Zhejiang University Science A, 2009, 10(6): 767-777.

@article{title="Dominant edge direction based fast intra mode decision in the H.264/AVC encoder",
author="Byeongdu LA, Minyoung EOM, Yoonsik CHOE",
journal="Journal of Zhejiang University Science A",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Dominant edge direction based fast intra mode decision in the H.264/AVC encoder
%A Byeongdu LA
%A Minyoung EOM
%A Yoonsik CHOE
%J Journal of Zhejiang University SCIENCE A
%V 10
%N 6
%P 767-777
%@ 1673-565X
%D 2009
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.A0820473

T1 - Dominant edge direction based fast intra mode decision in the H.264/AVC encoder
A1 - Byeongdu LA
A1 - Minyoung EOM
A1 - Yoonsik CHOE
J0 - Journal of Zhejiang University Science A
VL - 10
IS - 6
SP - 767
EP - 777
%@ 1673-565X
Y1 - 2009
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.A0820473

The h.264/AVC video coding standard uses an intra prediction mode with 4×4 and 16×16 blocks for luma and 8×8 blocks for chroma. This standard uses the rate distortion optimization (RDO) method to determine the best coding mode based on the compression performance and video quality. This method offers a large improvement in coding efficiency compared to other compression standards, but the computational complexity is greater due to the various intra prediction modes. This paper proposes a fast intra mode decision algorithm for real-time encoding of h.264/AVC based on the dominant edge direction (DED). The DED is extracted using pixel value summation and subtraction in the horizontal and vertical directions. By using the DED, three modes instead of nine are chosen for RDO calculation to decide on the best mode in the 4×4 luma block. For the 16×16 luma and the 8×8 chroma, only two modes are chosen instead of four. Experimental results show that the entire encoding time saving of the proposed algorithm is about 67% compared to the full intra search method with negligible loss of quality.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1] Bjontegaard, G., 2001. Calculation of Average PSNR Difference between RDCurve. ITU-T Q.6/16, Doc. VCEG-M33.

[2] Chen, J.W., Chang, C.H., Lin, C.C., Ouyang, Y.H., Guo, J.I., Wang, J.S., 2006. A Condition-based Intra Prediction Algorithm for H.264/AVC. Proc. IEEE Int. Conf. on Multimedia and Expo, p.1077-1080.

[3] Gao, S., Lu, T.J., 2006. An Improved Fast Mode Decision Algorithm in H.264 for Video Communications. Int. Symp. on Systems and Control in Aerospace and Astronautics, p.57-60.

[4] Kim, J., Jeon, K., Jeong, J., 2006. H.264 Intra mode decision for reducing complexity using directional masks and neighboring modes. LNCS, 4319:959-968.

[5] Lee, J., Jeon, B., 2004. Fast Mode Decision for H.264. Proc. IEEE Int. Conf. on Multimedia and Expo, 2:1131-1134.

[6] Liao, N., Quan, Z.Y., Men, A.D., 2005. Enhanced Fast Mode Decision Based on Edge Map and Motion Detail Analysis for H.264/JVT. IEEE Int. Workshop on VLSI Design and Video Technology, p.187-190.

[7] Meng, B., Au, O.C., 2003. Fast Intra-prediction Mode Selection for 4×4 Blocks in H.264. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, 3:389-392.

[8] Pan, F., Lin, X., Rahardja, S., Lim, K.P., Li, Z.G., Wu, D., Wu, S., 2005. Fast mode decision algorithm for intra prediction in H.264/AVC video coding. IEEE Trans. Circuits Syst. Video Technol., 15(7):813-822.

[9] Rehman, J., Zhang, Y., Kumarayapa, A., 2007. Fast Intra Prediction Mode Decision by Adaptively Selecting Fewer Number of Modes. Int. Conf. on Machine Learning and Cybernetics, p.2385-2389.

[10] Richardson, I.E.G., 2004. H.264 and MPEG-4 Video Compression. The Robert Gordon University, Aberdeen.

[11] Storey, D., Nasiopoulos, P., 2006. An Efficient Statistical Intra-mode Prediction Method for H.264. IEEE Int. Symp. on Signal Processing and Information Technology, p.803-806.

[12] Su, R., Liu, G.Z., Zhang, T.Y., 2006. Fast Mode Decision Algorithm for Intra Prediction in H.264/AVC. Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 2:921-924.

[13] Sullivan, G.J., Wiegand, T., 1998. Rate-distortion optimization for video compression. IEEE Signal Processing Mag., 15(6):74-90.

[14] Tseng, C.H., Wang, H.M., Yang, J.F., 2006. Enhanced intra-4×4 mode decision for H.264/AVC coders. IEEE Trans. Circuits Syst. Video Technol., 16(8):1027-1032.

[15] Wang, J.C., Wang, J.F., Yang, J.F., Chen, J.T., 2007. A fast mode decision algorithm and its VLSI design for H.264/AVC intra-prediction. IEEE Trans. Circuits Syst. Video Technol., 17(10):1414-1422.

[16] Wang, J.F., Wang, J.C., Chen, J.T., Tsai, A.C., Paul, A., 2006. A Novel Fast Algorithm for Intra Mode Decision in H.264/AVG Encoders. IEEE Int. Symp. on Circuits and Systems, p.3498-3501.

[17] Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A., 2003. Overview of the H.264/AVC video coding standard. IEEE Trans. Circuits Syst. Video Technol., 13(7):560-576.

[18] Xu, L.D., Lin, X.G., 2005. Fast Mode Decision for Inter Frames in H.264/AVC. IEEE Int. Symp. on Communications and Information Technology, 1:433-436.

[19] Zhang, Y.D., Dai, F., Lin, S.X., 2004. Fast 4×4 Intra-prediction Mode Selection for H.264. IEEE Int. Conf. on Multimedia and Expo, 2:1151-1154.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE