access icon free Pavement crack detection network based on pyramid structure and attention mechanism

Automatic detection of pavement crack is an important task for conducting road maintenance. However, as an important part of the intelligent transportation system, automatic pavement crack detection is challenging due to the poor continuity of cracks, the different width of cracks, and the low contrast between cracks and the surrounding pavement. This study proposes a novel pavement crack detection method based on an end-to-end trainable deep convolution neural network. The authors build the network using the encoder–decoder architecture and adopt a pyramid module to exploit global context information for the complex topology structures of cracks. Moreover, they introduce a spatial-channel combinational attention module into the encoder–decoder network for refining crack features. Further, the dilated convolution is used to reduce the loss of crack details due to the pooling operation in the encoder network. In addition, they introduce a lovász hinge loss function, which is suitable for small objects. They train the authors' network on the CRACK500 dataset and evaluate it on three pavement crack datasets. Among the methods they compare, their method can achieve the best experimental results.

Inspec keywords: cracks; structural engineering computing; intelligent transportation systems; maintenance engineering; convolutional neural nets; image recognition; roads; crack detection; object detection; learning (artificial intelligence)

Other keywords: attention mechanism; encoder-decoder architecture; intelligent transportation system; road maintenance; pavement crack datasets; complex topology structures; pooling operation; automatic pavement crack detection; end-to-end trainable deep convolution neural network; encoder-decoder network; surrounding pavement; crack features; CRACK500 dataset; spatial-channel combinational attention module; pavement crack detection network; pyramid structure

Subjects: Civil and mechanical engineering computing; Image recognition; Maintenance and reliability; Traffic engineering computing; Testing; Fracture mechanics and hardness (mechanical engineering); Mechanical engineering applications of IT; Neural computing techniques; Knowledge engineering techniques; Computer vision and image processing techniques

References

    1. 1)
      • 5. Maode, Y., Shaobo, B., Kun, X., et al: ‘Pavement crack detection and analysis for high-grade highway’. Int. Conf. on Electronic Measurement and Instruments, Hangzhou, 2007, pp. 548552.
    2. 2)
      • 10. Nguyen, T.S., Begot, S., Duculty, F., et al: ‘Free-form anisotropy: a new method for crack detection on pavement surface images’. IEEE Int. Conf. on Image Processing, Brussels, Belgium, 2011, pp. 10691072.
    3. 3)
      • 35. Feng, H., Xu, G., Guo, Y.: ‘Multi-scale classification network for road crack detection’, IET Intell. Transp. Syst., 2019, 13, (2), pp. 398405.
    4. 4)
      • 31. Subirats, P., Dumoulin, J., Legeay, V., et al: ‘Automation of pavement surface crack detection using the continuous wavelet transform’. Int. Conf. on Image Processing, Atlanta, GA, USA, 2006, pp. 30373040.
    5. 5)
      • 23. Zhai, M., Xiang, X., Zhang, R., et al: ‘Ad-net: attention guided network for optical flow estimation using dilated convolution’. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Brighton, UK, 2019, pp. 22072211.
    6. 6)
      • 2. Li, Q., Liu, X.: ‘Novel approach to pavement image segmentation based on neighboring difference histogram method’. Congress on Image and Signal Processing, Sanya, Hainan, 2008, vol. 2, pp. 726729.
    7. 7)
      • 4. Liu, F., Xu, G., Yang, Y., et al: ‘Novel approach to pavement cracking automatic detection based on segment extending’. Int. Symp. on Knowledge Acquisition and Modeling, Wuhan, 2008, pp. 610614.
    8. 8)
      • 37. Liu, W., Huang, Y., Li, Y., et al: ‘FPCNet: Fast Pavement Crack Detection Network Based on Encoder-Decoder Architecture’. arXiv e-prints, page arXiv:1907.02248, 2019.
    9. 9)
      • 25. Yu, F, Koltun, V.: ‘Multi-Scale Context Aggregation by Dilated Convolutions’, arXiv e-prints, page arXiv 1511.07122, 2015.
    10. 10)
      • 19. Lin, M., Chen, Q., Yan, S.: ‘Network In Network’. arXive-prints, page arXiv:1312.4400, 2013.
    11. 11)
      • 1. Kapela, R., Turkot, A., Rybarczyk, A., et al: ‘Asphalt surfaced pavement cracks detection based on histograms of oriented gradients’. Int. Conf. Mixed Design of Integrated Circuits Systems, Toruń, Poland, 2015, pp. 579584.
    12. 12)
      • 38. Shelhamer, E., Long, J., Darrell, T.: ‘Fully convolutional networks for semantic segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (4), pp. 640651.
    13. 13)
      • 9. Amhaz, R., Chambon, S., Idier, J., et al: ‘Automatic crack detection on two-dimensional pavement images: an algorithm based on minimal path selection’, IEEE Trans. Intell. Transp. Syst., 2016, 17, (10), pp. 27182729.
    14. 14)
      • 11. Oliveira, H., Correia, P.L.: ‘CrackIT - an image processing toolbox for crack detection and characterization’. IEEE Int. Conf. on Image Processing, 2014, pp. 798802.
    15. 15)
      • 26. Berman, M., Triki, A.R., Blaschko, M.B.: ‘The lovasz-softmax loss: a tractable surrogate for the optimization of the intersection-overunion measure in neural networks’. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 44134421.
    16. 16)
      • 27. Peng, L., Chao, W., Shuangmiao, L., et al: ‘Research on crack detection method of airport runway based on twice-threshold segmentation’. Int. Conf. on Instrumentation and Measurement, Computer, Communication and Control, Qinhuangdao, 2015, pp. 17161720.
    17. 17)
      • 3. Zhang, P., Li, H.: ‘Recognition of pavement surface crack’. Int. Conf. on Natural Computation, Yantai, 2010, vol. 7, pp. 33933396.
    18. 18)
      • 6. Qingbo, Z.: ‘Pavement crack detection algorithm based on image processing analysis’. Int. Conf. on Intelligent Human-Machine Systems and Cybernetics, Xi'an, 2016, vol. 01, pp. 1518.
    19. 19)
      • 22. He, A., Luo, C., Tian, X., et al: ‘A twofold siamese network for real-time object tracking’. IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 48344843.
    20. 20)
      • 17. Schmugge, S.J., Rice, L., Lindberg, J., et al: ‘Crack segmentation by leveraging multiple frames of varying illumination’. IEEE Winter Conf. on Applications of Computer Vision, Santa Rosa, CA, USA, 2017, pp. 10451053.
    21. 21)
      • 12. Zou, Q., Cao, Y., Li, Q., et al: ‘Cracktree: automatic crack detection from pavement images’, Pattern Recogn. Lett., 2012, 33, (3), pp. 227238.
    22. 22)
      • 34. Ai, D., Jiang, G., Siew Kei, L., et al: ‘Automatic pixel-level pavement crack detection using information of multi-scale neighborhoods’, IEEE Access, 2018, 6, pp. 2445224463.
    23. 23)
      • 36. Pauly, L., Hogg, D., Fuentes, R., et al: ‘Deeper networks for pavement crack detection’. Int. Symp. on Automation and Robotics in Construction, Taiwan, 2017, pp. 479485.
    24. 24)
      • 16. Yang, F., Zhang, L., Yu, S., et al: ‘Feature pyramid and hierarchical boosting network for pavement crack detection’, IEEE Trans. Intell. Transp. Syst., 2019, 21, (4), pp. 15251535.
    25. 25)
      • 8. Wei, N., Zhao, X., Dou, X.Y., et al: ‘Beamlet transform based pavement image crack detection’. Int. Conf. on Intelligent Computation Technology and Automation, Changsha, 2010, vol. 1, pp. 881883.
    26. 26)
      • 32. Zhou, J., Huang, P.S., Chiang, F.-P.: ‘Wavelet-based pavement distress detection and evaluation’, Opt. Eng., 2006, 45, (2), pp. 027007.1027007.10.
    27. 27)
      • 18. Badrinarayanan, V., Kendall, A., Cipolla, R.: ‘Segnet: a deep convolutional encoder-decoder architecture for image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (12), pp. 24812495.
    28. 28)
      • 30. Ayenu-Prah, A., Nii, A.-O.: ‘Evaluating pavement cracks with bidimensional empirical mode decomposition’. EURASIP J. Adv. Signal Process., 2008, 2008, (1), pp. 861701.1861701.7.
    29. 29)
      • 15. Zhang, L., Yang, F., Zhang, Y.D., et al: ‘Road crack detection using deep convolutional neural network’. IEEE Int. Conf. on Image Processing, Phoenix, AZ, USA, 2016, pp. 37083712.
    30. 30)
      • 20. He, K., Zhang, X., Ren, S., et al: ‘Spatial pyramid pooling in deep convolutional networks for visual recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 37, (9), pp. 19041916.
    31. 31)
      • 39. Zhou, B., Khosla, A., Lapedriza, A., et al: ‘Object Detectors Emerge in Deep Scene CNNs’. arXiv e-prints, page arXiv:1412.6856.2014.
    32. 32)
      • 14. Cha, Y.-J., Choi, W., Büyüköztürk, O.: ‘Deep learning-based crack damage detection using convolutional neural networks’, Comput. Aided Civil Infrastruct. Eng., 2017, 32, (5), pp. 361378.
    33. 33)
      • 29. Peng, L., Liu, J.: ‘Detection and analysis of large-scale WT blade surface cracks based on UAV-taken images’, IET Image Process, 2018, 12, (11), pp. 20592064.
    34. 34)
      • 24. Park, J., Woo, S., Lee, J.-Y., et al: ‘BAM: Bottleneck Attention Module’, arXiv e-prints, page arXiv:1807.06514, 2018.
    35. 35)
      • 7. Medina, R., Llamas, J., Zalama, E., et al: ‘Enhanced automatic detection of road surface cracks by combining 2d/3d image processing techniques’. IEEE Int. Conf. on Image Processing, Paris, France, 2014, pp. 778782.
    36. 36)
      • 33. Lin, Z., Zhao, X.: ‘Geometrical flow-guided fast beamlet transform for crack detection’, IET Image Process., 2018, 12, (3), pp. 328388.
    37. 37)
      • 28. Huang, W., Zhang, N.: ‘A novel road crack detection and identification method using digital image processing techniques’. Int. Conf. on Computing and Convergence Technology, Seoul, Republic of Korea, 2012, pp. 397400.
    38. 38)
      • 13. Shi, Y., Cui, L., Qi, Z., et al: ‘Automatic road crack detection using random structured forests’, IEEE Trans. Intell. Transp. Syst., 2016, 17, (12), pp. 34343445.
    39. 39)
      • 21. Hu, J., Shen, L., Albanie, S., et al: ‘Squeeze-and-excitation networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2019.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2019.0973
Loading

Related content

content/journals/10.1049/iet-ipr.2019.0973
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading