面向激光焊接缺陷识别的可解释性深度学习方法

doi:10.7527/S1000-6893.2021.24961

Abstract

Abstract: Laser welding has a wide range of applications in the aerospace industry. Vision-based laser welding defect detection is crucial to the improvement of product quality. To overcome the problem of poor explainability in current deep learning-based laser welding defect detection methods, a method of Class Activation Mapping by Incorporating Multiscale Features (MSF-CAM) is proposed. In the training phase, VGG16 is used as the backbone network, and supervisory information is applied to multiple scales to facilitate learning of multiscale features. In the testing phase, the activation maps of the output categories on multiple scales are superimposed and used as the basis for the judgment by the model. Integration of multiscale features not only enhances explainability of the model, but also improves accuracy of laser welding defect detection. The test results show that the accuracy of MSF-CAM on the test set is 98.12% and the time used by the method to identify a single image is 8.28 ms. In addition, MSF-CAM can provide more understandable explanation for the decision-making basis of the model from the perspective of such primary features as edge and contour.

Key words: laser welding, defect recognition, deep learning, explainability, multiscale

CLC Number:

V261.8

LIU Tianyuan, ZHENG Hangbin, YANG Changqi, BAO Jinsong, WANG Junliang, GU Jun. Explainable deep learning method for laser welding defect detection[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(4): 524961-524961.

References

[1] 王连庆, 胡雅楠, 车志刚, 等. 激光冲击强化7075铝合金熔焊接头的疲劳性能[J]. 航空学报, 2021, 42(5):524320. WANG L Q, HU Y N, CHE Z G, et al. Fatigue performance of laser shock processed fusion welded 7075 Al alloy[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(5):524320(in Chinese).
[2] MENG G, JI B, HAN H, et al. Design and simulation of an innovative cylinder fabricated by selective laser melting[J]. Chinese Journal of Aeronautics, 2019, 32(1):133-142.
[3] 金俊龙, 郭德伦, 刘琦, 等. 激光成形TC17钛合金线性摩擦焊接头组织与力学性能[J]. 焊接学报, 2019, 40(6):126-130, 166. JIN J L, GUO D L, LIU Q, et al. Microstructure and mechanical properties of linear friction welding joint of TC17 titanium alloy fabricated by laser forming[J]. Transactions of the China Welding Institution, 2019, 40(6):126-130, 166(in Chinese).
[4] 刘浩东, 胡芳友, 戴京涛, 等. 军机薄壁结构损伤超声跨态处理新工艺[J]. 焊接学报, 2016, 37(10):77-80, 133. LIU H D, HU F Y, DAI J T, et al. New technology of ultrasonic processing across different phases in laser welding for damage repairing of thin-walled structure in military aircraft[J]. Transactions of the China Welding Institution, 2016, 37(10):77-80, 133(in Chinese).
[5] ABIOYE T E, ZUHAILAWATI H, AIZAD S, et al. Geometrical, microstructural and mechanical characterization of pulse laser welded thin sheet 5052-H32 aluminium alloy for aerospace applications[J]. Transactions of Nonferrous Metals Society of China, 2019, 29(4):667-679.
[6] 夏令, 吴友发, 余海松, 等. 铝锂合金激光焊接接头组织与缺陷研究进展[J]. 航空制造技术, 2018, 61(S2):77-81, 86. XIA L, WU Y F, YU H S, et al. Research progress on microstructure and defects in Al-Li alloy laser welded joint[J]. Aeronautical Manufacturing Technology, 2018, 61(2):77-81, 86(in Chinese).
[7] 宫建锋, 李慧知, 李俐群, 等. 基于同轴图像传感的激光焊接过程质量监测技术[J]. 焊接学报, 2019, 40(1):37-42, 162. GONG J F, LI H Z, LI L Q, et al. Quality monitoring technology of laser welding process based on coaxial image sensing[J]. Transactions of the China Welding Institution, 2019, 40(1):37-42, 162(in Chinese).
[8] 贺钧婷, 陈宝华, 孙文卿, 等. 基于机器视觉的激光辅助焊接[J]. 激光杂志, 2018, 39(11):47-50. HE J T, CHEN B H, SUN W Q, et al. Laser assisted welding based on machine-vision[J]. Laser Journal, 2018, 39(11):47-50(in Chinese).
[9] LUO M, SHIN Y C. Vision-based weld pool boundary extraction and width measurement during keyhole fiber laser welding[J]. Optics and Lasers in Engineering, 2015, 64:59-70.
[10] YOU D Y, GAO X D, KATAYAMA S. Monitoring of high-power laser welding using high-speed photographing and image processing[J]. Mechanical Systems and Signal Processing, 2014, 49(1-2):39-52.
[11] TANG X, ZHONG P, ZHANG L L, et al. A new method to assess fiber laser welding quality of stainless steel 304 based on machine vision and hidden Markov models[J]. IEEE Access, 2020, 8:130633-130646.
[12] GVNTHER J, PILARSKI P M, HELFRICH G, et al. Intelligent laser welding through representation, prediction, and control learning:An architecture with deep neural networks and reinforcement learning[J]. Mechatronics, 2016, 34:1-11.
[13] ZHANG Z H, LI B, ZHANG W F, et al. Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks[J]. Journal of Manufacturing Systems, 2020, 54:348-360.
[14] ZHANG Y X, YOU D Y, GAO X D, et al. Online monitoring of welding status based on a DBN model during laser welding[J]. Engineering, 2019, 5(4):671-678.
[15] YANG Y T, YANG R Z, PAN L H, et al. A lightweight deep learning algorithm for inspection of laser welding defects on safety vent of power battery[J]. Computers in Industry, 2020, 123:103306.
[16] WANG Q Y, JIAO W H, WANG P, et al. A tutorial on deep learning-based data analytics in manufacturing through a welding case study[J]. Journal of Manufacturing Processes, 2021, 63:2-13.
[17] CAI W, WANG J Z, JIANG P, et al. Application of sensing techniques and artificial intelligence-based methods to laser welding real-time monitoring:A critical review of recent literature[J]. Journal of Manufacturing Systems, 2020, 57:1-18.
[18] WANG B C, HU S J, SUN L, et al. Intelligent welding system technologies:State-of-the-art review and perspectives[J]. Journal of Manufacturing Systems, 2020, 56:373-391.
[19] 纪守领, 李进锋, 杜天宇, 等. 机器学习模型可解释性方法、应用与安全研究综述[J]. 计算机研究与发展, 2019, 56(10):2071-2096. JI S L, LI J F, DU T Y, et al. Survey on techniques, applications and security of machine learning interpretability[J]. Journal of Computer Research and Development, 2019, 56(10):2071-2096(in Chinese).
[20] SUNG A H. Ranking importance of input parameters of neural networks[J]. Expert Systems With Applications, 1998, 15(3-4):405-411.
[21] RIBEIRO M T, SINGH S, GUESTRIN C. "Why should I trust You? ":Explaining the predictions of any classifier[C]//KDD'16:Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016:1135-1144.
[22] SIMONYAN K, VEDALDI A, ZISSERMAN A. Deep inside convolutional networks:Visualising image classification models and saliency maps[DB/OL]. arXiv preprint:1312.6034, 2013.
[23] ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016:2921-2929.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv preprint:1409.1556, 2014.
[25] DENG J, DONG W, SOCHER R, et al. ImageNet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2009:248-255.

Explainable deep learning method for laser welding defect detection

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	SU Lingfei, HUA Yongzhao, DONG Xiwang, REN Zhang. Human-UAV swarm multi-modal intelligent interaction methods [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 727001-727001.
[2]	LIU Jiaqi, FENG Yunwen, LU Cheng, XUE Xiaofeng, PAN Weihuang. Safety analysis of aero-engine operation based on intelligent neural network [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(9): 625375-625375.
[3]	ZHAO Liangyu, LI Dan, ZHAO Chenyue, JIANG Fei. Some achievements on detection methods of UAV autonomous landing markers [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(9): 25882-025882.
[4]	CAO Ming, WANG Peng, ZUO Hongfu, ZENG Haijun, SUN Jianzhong, YANG Weidong, WEI Fang, CHEN Xuefeng. Current status, challenges and opportunities of civil aero-engine diagnostics & health management Ⅱ: Comprehensive off-board diagnosis, life management and intelligent condition based MRO [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(9): 625574-625574.
[5]	KANG Yuxiang, CHEN Guo, WEI Xunkai, ZHOU Lei. Deep residual hedging network and its application in fault diagnosis of rolling bearings [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 625201-625201.
[6]	SHAO Yiwei, CHEN Jiayu, LIN Cuiying, WAN Cheng, GE Hongjuan, SHI Zhilong. Gearbox fault diagnosis with small training samples: An improved deep forest based method [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 625429-625429.
[7]	LIU Zichao, WANG Jiang, HE Shaoming, LI Yufei. A computational guidance algorithm for impact angle control based on predictor-corrector concept [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 325433-325433.
[8]	SUN Canfei, HUANG Linran, SHEN Yong. Fault diagnosis of helicopter planetary gear transmission system under complex operating conditions [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 625480-625480.
[9]	LYU Shuaishuai, YANG Yu, WANG Binwen, YIN Chenfei. A crack identification method of aircraft structure based on improved FaceNet [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(6): 525463-525463.
[10]	CHEN Bo, MENG Zheng, MA Chengyuan, XI Xin, WANG Xinxin, TAN Caiwang, SONG Xiaoguo. Welding properties and molten pool flow behavior of TC4 titanium alloy by oscillating galvanometer laser [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(4): 525223-525223.
[11]	KANG Shuo, KE Zhenzheng, WANG Xuan, ZHU Weidong. Detection method of defects in automatic fiber placement based on fusion of infrared and visible images [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(3): 425187-425187.
[12]	FENG Jicai. Research progress on dissimilar materials joining [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(2): 626413-626413.
[13]	ZOU Qi, YE Yiyun, JIAO Junke, WU Zhisheng, XU Zifa, ZHANG Wenwu. Performance analysis of carbon fiber reinforced thermalsetting composite-titanium alloy laser joint [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(2): 625037-625037.
[14]	ZHAN Qingliang, BAI Chunjin, ZHANG Ning, GE Yaojun. Feature extraction method of flow around airfoil based on time-history convolutional autoencoder [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(11): 526531-526531.
[15]	YI Jianmiao, DENG Feng, QIN Ning, LIU Xueqiang. Fast prediction of transonic flow field using deep learning method [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(11): 526747-526747.