Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon openaccess Unknown defect detection for printed circuit board based on multi-scale deep similarity measure method

Defect detection with high precision is of great significance for printed circuit board (PCB) fabrication. Due to the lack of priori knowledge of categories and shape features, detection of unknown defects faces greater challenges than that of common defects. Inspired by similarity measurement, this study proposes a multi-layer deep feature fusion method to calculate the similarity between template and defective circuit board. Compared with conventional methods which divide the whole detection into two independent parts of hand-designed features and similarity measurement, the authors end-to-end model is designed to combine these two parts for joint optimisation. First, the Siamese network is utilised as their backbone architecture for feature extraction of pairwise images. And then the spatial pyramid pooling network is incorporated into the feature maps of each convolutional module to fuse the multi-scale feature vectors. Finally, the discriminative feature embedding and similarity metric are obtained by using the contrastive loss during the training process. Experimental results show that the proposed model has better performance in detecting and locating unknown defects in bare PCB images than traditional similarity measurement methods. Moreover, our method is promising for further improvement of defect detection with less training image pairs and more accurate detection results.

References

    1. 1)
      • 2. Guo, F., Guan, S.: ‘Research of the machine vision based PCB defect inspection system’. Proc. Int. Conf. on Intelligence Science and Information Engineering, Wuhan, China, August 2011, pp. 472475.
    2. 2)
      • 31. Kong, T., Yao, A., Chen, Y., et al: ‘Hypernet: towards accurate region proposal generation and joint object detection’. Proc. Int. Conf. in Computer Vision Pattern Recognition, Las Vegas, NV, USA, June 2016, pp. 845853.
    3. 3)
      • 9. Kim, J.H., Cho, H.S.: ‘Neural network-based inspection of solder joints using a circular illumination’, Image Vis. Comput,1995, 13, (6), pp. 479490.
    4. 4)
      • 5. Ren, S., Lu, L., Zhao, L., et al: ‘Circuit board defect detection based on image processing’. Proc. Int. Congress on Image and Signal Processing (CISP), Shenyang, China, October 2015, pp. 899903.
    5. 5)
      • 39. Gaidhane, V.H., Hote, Y.V., Singh, V.: ‘An efficient similarity measure approach for PCB surface defect detection’. Pattern Anal. Appl., 2018, 21, (1), pp. 277289.
    6. 6)
      • 20. Roh, B., Yoon, C., Ryu, Y., et al: ‘A neural network approach to defect classification on printed circuit boards’, J. Inst. Contr., Robot. Syst.,2001, 67, (10), pp. 16211626.
    7. 7)
      • 18. Noda, K., Yamaguchi, Y., Nakadai, K.: ‘Audio-visual speech recognition using deep learning’, Appl. Intell., 2015, 42, (4), pp. 722737.
    8. 8)
      • 37. Davis, J., Goadrich, M.: ‘The relationship between precision-recall and roc curves’. Proc. Int. Conf. on Machine learning (ICML), New York, NY, USA, 2006, pp. 233240.
    9. 9)
      • 22. Caldo, R.B.: ‘ANN diagnosis for defect detection and classification in two-layer printed circuit boards using supervised back-propagation algorithm’, Adv. Comput. Commun. Eng. Technol.,2016, pp. 577593.
    10. 10)
      • 12. Hao, W., Xianmin, Z., Yongcong, K., et al: ‘Solder joint inspection based on neural network combined with genetic algorithm’, Int. J. Light Electr. Opt., 2013, 124, (20), pp. 41104116.
    11. 11)
      • 16. Zeiler, M.D., Fergus, R.: ‘Visualizing and understanding convolutional networks’. Proc. Int. Conf. In European Conf. on Computer Vision, Zurich, Switzerland, September 2014, pp. 818833.
    12. 12)
      • 8. Inoue, H., Iwahori, Y., Kijsirikul, B., et al: ‘SVM based defect classification of electronic board using bag of keypoints’. Proc. Int. Techn. Conf. on Circuits Systems, Computers and Communication, Seoul, Korea, June 2015, pp. 3134.
    13. 13)
      • 11. Ong, T.Y., Samad, Z., Ratnam, M.M.: ‘Solder joint inspection with multi-angle imaging and an artificial neural network’, Int. J. Adv. Manuf. Technol.,2008, 38, (5-6), pp. 455462.
    14. 14)
      • 40. Tsai, D.M., Yang, R.H.: ‘An eigenvalue-based similarity measure and its application in defect detection’. Image Vis. Comput., 2005, 23, (12), pp. 10941101.
    15. 15)
      • 1. Moganti, M., Ercal, F., Dagli, C.H.: ‘Automatic PCB inspection algorithms: a survey’, Comput. Vis. Image Underst., 1996, 63, (2), pp. 287313.
    16. 16)
      • 3. Wu, W.Y., Wang, M.J.J., Liu, C.M.: ‘Automated inspection of printed circuit boards through machine vision’, Comput. Ind.,1996, 28, (2), pp. 103111.
    17. 17)
      • 4. Putera, S.H.I., Dzafaruddin, S.F., Mohamad, M.: ‘MATLAB based defect detection and classification of printed circuit board’. Proc. Int. Conf. on Digital Information and Communication Technology and it's Applications, Bangkok, Thailand, May 2012, pp. 115119.
    18. 18)
      • 23. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘Imagenet classification with deep convolutional neural networks’. Proc. Int. Conf. in Advances in Neural Information Processing Systems, Lake, Tahoe, USA, December 2012, pp. 10971105.
    19. 19)
      • 21. Bay, H., Tuytelaars, T., Van Gool, L., et al: ‘Surf: speeded up robust features’. Proc. Int. Conf. In European Conf. on Computer Vision, Heidelberg, Berlin, May 2006, pp. 404417.
    20. 20)
      • 17. LeCun, Y., Bengio, Y., Hinton, G.: ‘Deep learning’, Nature, 2015, 521, (7553), p. 436.
    21. 21)
      • 25. Szegedy, C., Liu, W., Jia, Y.Q., et al: ‘Going deeper with convolutions’. Proc. Int. Conf. on Computer Vision and Pattern Recognition, Boston, MA, USA, June 2015, pp. 19.
    22. 22)
      • 35. Zeiler, M. D.: ‘ADADELTA: an adaptive learning rate method’, arXiv preprint arXiv:1212.5701, 2012.
    23. 23)
      • 33. Hariharan, B., Arbelaez, P., Girshick, R., et al: ‘Hypercolumns for object segmentation and fine-grained localization’. Proc. Int. Conf. in Computer Vision Pattern Recognition, Boston, MA, USA, June 2015, pp. 447456.
    24. 24)
      • 28. Ibrahim, I., Abdul Rahman, S., Abu Bakar, S., et al: ‘A printed circuit board inspection system with defect classification capability’, Int. J. Innov. Manage. Inf. Prod., 2012, 3, (1).
    25. 25)
      • 19. Wu, G., Lu, W., Gao, G., et al: ‘Regional deep learning model for visual tracking’, Neural Comput., 2016, 175, pp. 310323.
    26. 26)
      • 27. Moganti, M., Ercal, F.: ‘Automatic PCB inspection systems’, IEEE Potent,1995, 14, (3), pp. 610.
    27. 27)
      • 24. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large scale image recognition’, arXiv preprint arXiv:1409.1556, 2014.
    28. 28)
      • 7. Kumar, S., Iwahori, Y., Bhuyan, M.K.: ‘PCB defect classification using logical combination of segmented copper and non-copper part’. Proc. Int. Conf. on Computer Vision and Image Processing, Singapore, 2017, pp. 523532.
    29. 29)
      • 6. Anu, A., Anouncia, M., Rajaji, L.: ‘Automatic visual inspection of PCB using CAD information’. Proc. Int. Conf. on Radar, Communication and Computing (ICRCC), Tamil Nadu, India, December 2012, pp. 2122.
    30. 30)
      • 15. Yun, T.S., Sim, K.J., Kim, H.J.: ‘Support vector machine-based inspection of solder joints using circular illumination’, Electr. Lett.,2000, 36, (11), pp. 949951.
    31. 31)
      • 38. Everingham, M., Van Gool, L., Williams, C.K., et al: ‘The Pascal visual object classes (VOC) challenge’, Int. J. Comput. Vis.,2010, 88, (2), pp. 303338.
    32. 32)
      • 26. Zhang, C., Shi, W., Li, X., et al: ‘Improved bare PCB defect detection approach based on deep feature learning’, J. Eng.,2018, 16, pp. 14151420.
    33. 33)
      • 34. Weibo, H., Peng, W.: ‘A PCB dataset for defects detection and classification’, arXiv preprint arXiv:1901.08204, 2019.
    34. 34)
      • 13. Luo, B., Zhang, Y., Yu, G., et al: ‘ANN ensembles based machine vision inspection for solder joints’. Proc. Int. Conf. on Control and Automation, Guangzhou, China, June 2007, pp. 31113115.
    35. 35)
      • 29. LeCun, Y., Boser, B., Denker, J. S., et al: ‘Backpropagation applied to handwritten zip code recognition’, Neural Comput., 1989, 1, (4), pp. 541551.
    36. 36)
      • 14. Kim, T.H., Cho, T.H., Moon, Y.S., et al: ‘Visual inspection system for the classification of solder joints’, Pattern Recognit., 1999, 32, (4), pp. 565575.
    37. 37)
      • 10. Acciani, G., Brunetti, G., Fornarelli, G.: ‘Application of neural networks in optical inspection and classification of solder joints in surface mount technology’, IEEE Trans. Ind. Inf.,2006, 2, (3), pp. 200209.
    38. 38)
      • 32. Bell, S., Zitnick, C.L., Bala, K., et al: ‘Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks’. Proc. Int. Conf. in Computer Vision Pattern Recognition, Boston, MA, USA, June 2015, pp. 28742883.
    39. 39)
      • 36. Lin, T.Y., Cui, Y., Belongie, S., et al: ‘Learning deep representations for ground-to-aerial geolocalization’. Proc. Int. Conf. on Computer Vision and Pattern Recognition, Boston, MA, USA, June 2015, pp. 50075015.
    40. 40)
      • 30. Chandrasekhar, V., Lin, J., Morere, O., et al: ‘A practical guide to CNNs and Fisher vectors for image instance retrieval’, Int. J. Signal Process,2016, 128, pp. 426439.
http://iet.metastore.ingenta.com/content/journals/10.1049/joe.2019.1188
Loading

Related content

content/journals/10.1049/joe.2019.1188
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address