Abstract: Defect inspection, also known as defect detection, is significant in mobile screen quality control. There are some challenging issues brought by the characteristics of screen defects, including the following: (1) the problem of interclass similarity and intraclass variation, (2) the difficulty in distinguishing low contrast, tiny-sized, or incomplete defects, and (3) the modeling of category dependencies for multi-label images. To solve these problems, a graph reasoning module, stacked on a classification module, is proposed to expand the feature dimension and improve low-quality image features by exploiting category-wise dependency, image-wise relations, and interactions between them. To further improve the classification performance, the classifier of the classification module is redesigned as a cosine similarity function. With the help of contrastive learning, the classification module can better initialize the category-wise graph of the reasoning module. Experiments on the mobile screen defect dataset show that our two-stage network achieves the following best performances: 97.7% accuracy and 97.3% F-measure. This proves that the proposed approach is effective in industrial applications.

用于手机屏缺陷检测的基于图的两阶段分类网络

[1]Bottou L, 2010. Large-scale machine learning with stochastic gradient descent. Proc 19^th Int Conf on Computational Statistics, p.177-186.

[2]Chen T, Kornblith S, Norouzi M, et al., 2020. A simple framework for contrastive learning of visual representations. Proc 37^th Int Conf on Machine Learning, p.1597-1607.

[3]Chen ZM, Wei XS, Wang P, et al., 2019. Multi-label image recognition with graph convolutional networks. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.5177-5186.

[4]Gidaris S, Komodakis N, 2018. Dynamic few-shot visual learning without forgetting. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.4367-4375.

[5]Haurum JB, Moeslund TB, 2021. Sewer-ML: a multi-label sewer defect classification dataset and benchmark. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.13451-13462.

[6]He KM, Zhang XY, Ren SQ, et al., 2016. Deep residual learning for image recognition. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.770-778.

[7]He KM, Fan HQ, Wu YX, et al., 2020. Momentum contrast for unsupervised visual representation learning. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9726-9735.

[8]Hjelm RD, Fedorov A, Lavoie-Marchildon S, et al., 2019. Learning deep representations by mutual information estimation and maximization. Proc 7^th Int Conf on Learning Representations.

[9]Khosla P, Teterwak P, Wang C, et al., 2020. Supervised contrastive learning. Proc 34^th Conf on Neural Information Processing Systems, p.18661-18673.

[10]Kong YH, Liu X, Zhao ZB, et al., 2022. Bolt defect classification algorithm based on knowledge graph and feature fusion. Energy Rep, 8(Suppl 1):856-863.

[11]Lei J, Gao X, Feng ZL, et al., 2018. Scale insensitive and focus driven mobile screen defect detection in industry. Neurocomputing, 294:72-81.

[12]Li CS, Zhang XM, Huang YJ, et al., 2020. A novel algorithm for defect extraction and classification of mobile phone screen based on machine vision. Comput Ind Eng, 146:106530.

[13]Lu Y, Ma L, Jiang HQ, 2020. A light CNN model for defect detection of LCD. In: Hung JC, Yen NY, Chang JW (Eds.), Frontier Computing. Springer, Singapore, p.10-19.

[14]Park JY, Hwang Y, Lee D, et al., 2020. MarsNet: multi-label classification network for images of various sizes. IEEE Access, 8:21832-21846.

[15]Paszke A, Gross S, Chintala S, et al., 2017. Automatic differentiation in PyTorch. Proc 31^st Conf on Neural Information Processing Systems.

[16]Simonyan K, Zisserman A, 2015. Very deep convolutional networks for large-scale image recognition. Proc 3^rd Int Conf on Learning Representations.

[17]Szegedy C, Vanhoucke V, Ioffe S, et al., 2016. Rethinking the inception architecture for computer vision. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2818-2826.

[18]Wang T, Zhang C, Ding RW, et al., 2021. Mobile phone surface defect detection based on improved faster R-CNN. Proc 25^th Int Conf on Pattern Recognition, p.9371-9377.

[19]Wang Y, He DL, Li F, et al., 2020. Multi-label classification with label graph superimposing. Proc AAAI Conf Artif Intell, 34(7):12265-12272. doi:

[20]Wang YC, Gao L, Li XY, et al., 2021a. A new graph-based method for class imbalance in surface defect recognition. IEEE Trans Instrum Meas, 70:5007816.

[21]Wang YC, Gao L, Gao YP, et al., 2021b. A new graph-based semi-supervised method for surface defect classification. Rob Comput Integr Manuf, 68:102083.

[22]Wei B, Hao KR, Gao L, et al., 2021. Bioinspired visual-integrated model for multilabel classification of textile defect images. IEEE Trans Cognit Dev Syst, 13(3):503-513.

[23]Wu ZR, Xiong YJ, Yu SX, et al., 2018. Unsupervised feature learning via non-parametric instance discrimination. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3733-3742.

[24]Xiao WW, Song KC, Liu J, et al., 2022. Graph embedding and optimal transport for few-shot classification of metal surface defect. IEEE Trans Instrum Meas, 71:5010310.

[25]Xu H, Jiang CH, Liang XD, et al., 2019. Reasoning-RCNN: unifying adaptive global reasoning into large-scale object detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.6412-6421.

[26]Yuan ZC, Zhang ZT, Su H, et al., 2018. Vision-based defect detection for mobile phone cover glass using deep neural networks. Int J Precis Eng Manuf, 19(6):801-810.

[27]Zhang JB, Su H, Zou W, et al., 2021. CADN: a weakly supervised learning-based category-aware object detection network for surface defect detection. Patt Recogn, 109:107571.

[28]Zhao JW, Yan K, Zhao YF, et al., 2021. Transformer-based dual relation graph for multi-label image recognition. Proc IEEE/CVF Int Conf on Computer Vision, p.163-172.

[29]Zhou BL, Khosla A, Lapedriza A, et al., 2016. Learning deep features for discriminative localization. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2921-2929.

[30]Zhu Y, Ding RW, Huang WB, et al., 2022. HMFCA-Net: hierarchical multi-frequency based channel attention net for mobile phone surface defect detection. Patt Recogn Lett, 153:118-125.