Explainable deep learning method for laser welding defect detection

LIU Tianyuan; ZHENG Hangbin; YANG Changqi; BAO Jinsong; WANG Junliang; GU Jun

doi:10.7527/S1000-6893.2021.24961

ACTA AERONAUTICAET ASTRONAUTICA SINICA >

2022 , Vol. 43 >Issue 4: 524961 - 524961

DOI: https://doi.org/10.7527/S1000-6893.2021.24961

Articles

Explainable deep learning method for laser welding defect detection

LIU Tianyuan ,
ZHENG Hangbin ,
YANG Changqi ,
BAO Jinsong ,
WANG Junliang ,
GU Jun

Expand

1. Institute of Intelligent Manufacturing, College of Mechanical Engineering, Donghua University, Shanghai 201600, China;
2. Shanghai Spaceflight Precision Machinery Institute, Shanghai 201600, China;
3. Shanghai Institute of Laser Technology, Shanghai 200235, China

Received date: 2020-11-11

Revised date: 2020-11-30

Online published: 2021-02-24

Supported by

National Natural Science Foundation of China (51905091); Fundamental Research Funds for the Central Universities and Graduate Student Innovation Fund of Donghua University (CUSF-DH-D-2020053)

Fold

Abstract

Laser welding has a wide range of applications in the aerospace industry. Vision-based laser welding defect detection is crucial to the improvement of product quality. To overcome the problem of poor explainability in current deep learning-based laser welding defect detection methods, a method of Class Activation Mapping by Incorporating Multiscale Features (MSF-CAM) is proposed. In the training phase, VGG16 is used as the backbone network, and supervisory information is applied to multiple scales to facilitate learning of multiscale features. In the testing phase, the activation maps of the output categories on multiple scales are superimposed and used as the basis for the judgment by the model. Integration of multiscale features not only enhances explainability of the model, but also improves accuracy of laser welding defect detection. The test results show that the accuracy of MSF-CAM on the test set is 98.12% and the time used by the method to identify a single image is 8.28 ms. In addition, MSF-CAM can provide more understandable explanation for the decision-making basis of the model from the perspective of such primary features as edge and contour.

Key words： laser welding; defect recognition; deep learning; explainability; multiscale

Cite this article

LIU Tianyuan , ZHENG Hangbin , YANG Changqi , BAO Jinsong , WANG Junliang , GU Jun . Explainable deep learning method for laser welding defect detection[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022 , 43(4) : 524961 -524961 . DOI: 10.7527/S1000-6893.2021.24961

References

[1] 王连庆, 胡雅楠, 车志刚, 等. 激光冲击强化7075铝合金熔焊接头的疲劳性能[J]. 航空学报, 2021, 42(5):524320. WANG L Q, HU Y N, CHE Z G, et al. Fatigue performance of laser shock processed fusion welded 7075 Al alloy[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(5):524320(in Chinese).
[2] MENG G, JI B, HAN H, et al. Design and simulation of an innovative cylinder fabricated by selective laser melting[J]. Chinese Journal of Aeronautics, 2019, 32(1):133-142.
[3] 金俊龙, 郭德伦, 刘琦, 等. 激光成形TC17钛合金线性摩擦焊接头组织与力学性能[J]. 焊接学报, 2019, 40(6):126-130, 166. JIN J L, GUO D L, LIU Q, et al. Microstructure and mechanical properties of linear friction welding joint of TC17 titanium alloy fabricated by laser forming[J]. Transactions of the China Welding Institution, 2019, 40(6):126-130, 166(in Chinese).
[4] 刘浩东, 胡芳友, 戴京涛, 等. 军机薄壁结构损伤超声跨态处理新工艺[J]. 焊接学报, 2016, 37(10):77-80, 133. LIU H D, HU F Y, DAI J T, et al. New technology of ultrasonic processing across different phases in laser welding for damage repairing of thin-walled structure in military aircraft[J]. Transactions of the China Welding Institution, 2016, 37(10):77-80, 133(in Chinese).
[5] ABIOYE T E, ZUHAILAWATI H, AIZAD S, et al. Geometrical, microstructural and mechanical characterization of pulse laser welded thin sheet 5052-H32 aluminium alloy for aerospace applications[J]. Transactions of Nonferrous Metals Society of China, 2019, 29(4):667-679.
[6] 夏令, 吴友发, 余海松, 等. 铝锂合金激光焊接接头组织与缺陷研究进展[J]. 航空制造技术, 2018, 61(S2):77-81, 86. XIA L, WU Y F, YU H S, et al. Research progress on microstructure and defects in Al-Li alloy laser welded joint[J]. Aeronautical Manufacturing Technology, 2018, 61(2):77-81, 86(in Chinese).
[7] 宫建锋, 李慧知, 李俐群, 等. 基于同轴图像传感的激光焊接过程质量监测技术[J]. 焊接学报, 2019, 40(1):37-42, 162. GONG J F, LI H Z, LI L Q, et al. Quality monitoring technology of laser welding process based on coaxial image sensing[J]. Transactions of the China Welding Institution, 2019, 40(1):37-42, 162(in Chinese).
[8] 贺钧婷, 陈宝华, 孙文卿, 等. 基于机器视觉的激光辅助焊接[J]. 激光杂志, 2018, 39(11):47-50. HE J T, CHEN B H, SUN W Q, et al. Laser assisted welding based on machine-vision[J]. Laser Journal, 2018, 39(11):47-50(in Chinese).
[9] LUO M, SHIN Y C. Vision-based weld pool boundary extraction and width measurement during keyhole fiber laser welding[J]. Optics and Lasers in Engineering, 2015, 64:59-70.
[10] YOU D Y, GAO X D, KATAYAMA S. Monitoring of high-power laser welding using high-speed photographing and image processing[J]. Mechanical Systems and Signal Processing, 2014, 49(1-2):39-52.
[11] TANG X, ZHONG P, ZHANG L L, et al. A new method to assess fiber laser welding quality of stainless steel 304 based on machine vision and hidden Markov models[J]. IEEE Access, 2020, 8:130633-130646.
[12] GVNTHER J, PILARSKI P M, HELFRICH G, et al. Intelligent laser welding through representation, prediction, and control learning:An architecture with deep neural networks and reinforcement learning[J]. Mechatronics, 2016, 34:1-11.
[13] ZHANG Z H, LI B, ZHANG W F, et al. Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks[J]. Journal of Manufacturing Systems, 2020, 54:348-360.
[14] ZHANG Y X, YOU D Y, GAO X D, et al. Online monitoring of welding status based on a DBN model during laser welding[J]. Engineering, 2019, 5(4):671-678.
[15] YANG Y T, YANG R Z, PAN L H, et al. A lightweight deep learning algorithm for inspection of laser welding defects on safety vent of power battery[J]. Computers in Industry, 2020, 123:103306.
[16] WANG Q Y, JIAO W H, WANG P, et al. A tutorial on deep learning-based data analytics in manufacturing through a welding case study[J]. Journal of Manufacturing Processes, 2021, 63:2-13.
[17] CAI W, WANG J Z, JIANG P, et al. Application of sensing techniques and artificial intelligence-based methods to laser welding real-time monitoring:A critical review of recent literature[J]. Journal of Manufacturing Systems, 2020, 57:1-18.
[18] WANG B C, HU S J, SUN L, et al. Intelligent welding system technologies:State-of-the-art review and perspectives[J]. Journal of Manufacturing Systems, 2020, 56:373-391.
[19] 纪守领, 李进锋, 杜天宇, 等. 机器学习模型可解释性方法、应用与安全研究综述[J]. 计算机研究与发展, 2019, 56(10):2071-2096. JI S L, LI J F, DU T Y, et al. Survey on techniques, applications and security of machine learning interpretability[J]. Journal of Computer Research and Development, 2019, 56(10):2071-2096(in Chinese).
[20] SUNG A H. Ranking importance of input parameters of neural networks[J]. Expert Systems With Applications, 1998, 15(3-4):405-411.
[21] RIBEIRO M T, SINGH S, GUESTRIN C. "Why should I trust You? ":Explaining the predictions of any classifier[C]//KDD'16:Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016:1135-1144.
[22] SIMONYAN K, VEDALDI A, ZISSERMAN A. Deep inside convolutional networks:Visualising image classification models and saliency maps[DB/OL]. arXiv preprint:1312.6034, 2013.
[23] ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016:2921-2929.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv preprint:1409.1556, 2014.
[25] DENG J, DONG W, SOCHER R, et al. ImageNet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2009:248-255.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References