JZUS - Journal of Zhejiang University SCIENCE

Frontiers of Information Technology & Electronic Engineering

Accepted manuscript available online (unedited version)

A graph-based two-stage classification network for mobile screen defect inspection

Author(s): Chaofan ZHOU, Meiqin LIU, Senlin ZHANG, Ping WEI, Badong CHEN
Affiliation(s): State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China; more
Corresponding email(s): zhouchaofan@zju.edu.cn, liumeiqin@zju.edu.cn, slzhang@zju.edu.cn, pingwei@mail.xjtu.edu.cn, chenbd@mail.xjtu.edu.cn
Key Words: Graph-based methods; Multi-label classification; Mobile screen defects; Neural networks

Share this article to： More <<< Previous Paper \|Next Paper >>>

Abstract: Defect inspection, also known as defect detection, is significant in mobile screen quality control. There are some challenging issues brought by the characteristics of screen defects, including the following: (1) the problem of interclass similarity and intraclass variation, (2) the difficulty in distinguishing low contrast, tiny-sized, or incomplete defects, and (3) the modeling of category dependencies for multi-label images. To solve these problems, a graph reasoning module, stacked on a classification module, is proposed to expand the feature dimension and improve low-quality image features by exploiting category-wise dependency, image-wise relations, and interactions between them. To further improve the classification performance, the classifier of the classification module is redesigned as a cosine similarity function. With the help of contrastive learning, the classification module can better initialize the category-wise graph of the reasoning module. Experiments on the mobile screen defect dataset show that our two-stage network achieves the following best performances: 97.7% accuracy and 97.3% F-measure. This proves that the proposed approach is effective in industrial applications.

用于手机屏缺陷检测的基于图的两阶段分类网络

周超凡^1,2，刘妹琴^3,2,1，张森林^1,2，魏平³，陈霸东³
¹浙江大学工业控制技术国家重点实验室，中国杭州市，310027
²浙江大学电气工程学院，中国杭州市，310027
³西安交通大学人工智能与机器人研究所，中国西安市，710049
摘要：缺陷检测是手机屏质量控制的重要环节。手机屏缺陷的特性带来了一些具有挑战性的问题，包括：（1）类间相似性和类内差异性；（2）低对比度、微小尺寸或不完整缺陷的识别带来的困难；（3）针对多标签图像的类别相关性建模。为了解决这些问题，本文提出一种图推理模块，它可以堆放在常规的分类模块上。该推理模块利用类别间的依赖性、图像间的关系以及类别图像之间的相互作用来扩展特征维度，并且达到改进低质量图像特征的目的。为了进一步提高分类性能，分类模块的分类器被设计为一个余弦相似度函数。在对比学习的帮助下，分类模块可以更好地初始化推理模块的类别图。在手机屏缺陷数据集上的实验表明，所提出的两阶段网络取得了最佳性能：准确率为97.7％，F-measure为97.3％。这证明了本文所提出的方法在工业应用中是有效的。

关键词组：基于图的方法；多标签分类；手机屏缺陷；神经网络

Reference

[1]Bottou L, 2010. Large-scale machine learning with stochastic gradient descent. Proc 19^th Int Conf on Computational Statistics, p.177-186.

[2]Chen T, Kornblith S, Norouzi M, et al., 2020. A simple framework for contrastive learning of visual representations. Proc 37^th Int Conf on Machine Learning, p.1597-1607.

[3]Chen ZM, Wei XS, Wang P, et al., 2019. Multi-label image recognition with graph convolutional networks. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.5177-5186.

[4]Gidaris S, Komodakis N, 2018. Dynamic few-shot visual learning without forgetting. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.4367-4375.

[5]Haurum JB, Moeslund TB, 2021. Sewer-ML: a multi-label sewer defect classification dataset and benchmark. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.13451-13462.

[6]He KM, Zhang XY, Ren SQ, et al., 2016. Deep residual learning for image recognition. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.770-778.

[7]He KM, Fan HQ, Wu YX, et al., 2020. Momentum contrast for unsupervised visual representation learning. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9726-9735.

[8]Hjelm RD, Fedorov A, Lavoie-Marchildon S, et al., 2019. Learning deep representations by mutual information estimation and maximization. Proc 7^th Int Conf on Learning Representations.

[9]Khosla P, Teterwak P, Wang C, et al., 2020. Supervised contrastive learning. Proc 34^th Conf on Neural Information Processing Systems, p.18661-18673.

[10]Kong YH, Liu X, Zhao ZB, et al., 2022. Bolt defect classification algorithm based on knowledge graph and feature fusion. Energy Rep, 8(Suppl 1):856-863.

[11]Lei J, Gao X, Feng ZL, et al., 2018. Scale insensitive and focus driven mobile screen defect detection in industry. Neurocomputing, 294:72-81.

[12]Li CS, Zhang XM, Huang YJ, et al., 2020. A novel algorithm for defect extraction and classification of mobile phone screen based on machine vision. Comput Ind Eng, 146:106530.

[13]Lu Y, Ma L, Jiang HQ, 2020. A light CNN model for defect detection of LCD. In: Hung JC, Yen NY, Chang JW (Eds.), Frontier Computing. Springer, Singapore, p.10-19.

[14]Park JY, Hwang Y, Lee D, et al., 2020. MarsNet: multi-label classification network for images of various sizes. IEEE Access, 8:21832-21846.

[15]Paszke A, Gross S, Chintala S, et al., 2017. Automatic differentiation in PyTorch. Proc 31^st Conf on Neural Information Processing Systems.

[16]Simonyan K, Zisserman A, 2015. Very deep convolutional networks for large-scale image recognition. Proc 3^rd Int Conf on Learning Representations.

[17]Szegedy C, Vanhoucke V, Ioffe S, et al., 2016. Rethinking the inception architecture for computer vision. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2818-2826.

[18]Wang T, Zhang C, Ding RW, et al., 2021. Mobile phone surface defect detection based on improved faster R-CNN. Proc 25^th Int Conf on Pattern Recognition, p.9371-9377.

[19]Wang Y, He DL, Li F, et al., 2020. Multi-label classification with label graph superimposing. Proc AAAI Conf Artif Intell, 34(7):12265-12272. doi:

[20]Wang YC, Gao L, Li XY, et al., 2021a. A new graph-based method for class imbalance in surface defect recognition. IEEE Trans Instrum Meas, 70:5007816.

[21]Wang YC, Gao L, Gao YP, et al., 2021b. A new graph-based semi-supervised method for surface defect classification. Rob Comput Integr Manuf, 68:102083.

[22]Wei B, Hao KR, Gao L, et al., 2021. Bioinspired visual-integrated model for multilabel classification of textile defect images. IEEE Trans Cognit Dev Syst, 13(3):503-513.

[23]Wu ZR, Xiong YJ, Yu SX, et al., 2018. Unsupervised feature learning via non-parametric instance discrimination. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3733-3742.

[24]Xiao WW, Song KC, Liu J, et al., 2022. Graph embedding and optimal transport for few-shot classification of metal surface defect. IEEE Trans Instrum Meas, 71:5010310.

[25]Xu H, Jiang CH, Liang XD, et al., 2019. Reasoning-RCNN: unifying adaptive global reasoning into large-scale object detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.6412-6421.

[26]Yuan ZC, Zhang ZT, Su H, et al., 2018. Vision-based defect detection for mobile phone cover glass using deep neural networks. Int J Precis Eng Manuf, 19(6):801-810.

[27]Zhang JB, Su H, Zou W, et al., 2021. CADN: a weakly supervised learning-based category-aware object detection network for surface defect detection. Patt Recogn, 109:107571.

[28]Zhao JW, Yan K, Zhao YF, et al., 2021. Transformer-based dual relation graph for multi-label image recognition. Proc IEEE/CVF Int Conf on Computer Vision, p.163-172.

[29]Zhou BL, Khosla A, Lapedriza A, et al., 2016. Learning deep features for discriminative localization. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2921-2929.

[30]Zhu Y, Ding RW, Huang WB, et al., 2022. HMFCA-Net: hierarchical multi-frequency based channel attention net for mobile phone surface defect detection. Patt Recogn Lett, 153:118-125.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

- Go to

用于手机屏缺陷检测的基于图的两阶段分类网络

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference