面向激光焊接缺陷识别的可解释性深度学习方法

刘天元; 郑杭彬; 杨长祺; 鲍劲松; 汪俊亮; 顾俊

doi:10.7527/S1000-6893.2021.24961

航空学报 >

2022 , Vol. 43 >Issue 4: 524961 - 524961

DOI: https://doi.org/10.7527/S1000-6893.2021.24961

论文

面向激光焊接缺陷识别的可解释性深度学习方法

刘天元 ,
郑杭彬 ,
杨长祺 ,
鲍劲松 ,
汪俊亮 ,
顾俊

展开

1. 东华大学机械工程学院智能制造研究所, 上海 201600;
2. 上海航天精密机械研究所, 上海 201600;
3. 上海市激光技术研究所, 上海 200235

收稿日期: 2020-11-11

修回日期: 2020-11-30

网络出版日期: 2021-02-24

基金资助

国家自然科学基金（51905091）；中央高校基本科研业务费专项资金-东华大学研究生创新基金（CUSF-DH-D-2020053）

收起

Explainable deep learning method for laser welding defect detection

LIU Tianyuan ,
ZHENG Hangbin ,
YANG Changqi ,
BAO Jinsong ,
WANG Junliang ,
GU Jun

Expand

1. Institute of Intelligent Manufacturing, College of Mechanical Engineering, Donghua University, Shanghai 201600, China;
2. Shanghai Spaceflight Precision Machinery Institute, Shanghai 201600, China;
3. Shanghai Institute of Laser Technology, Shanghai 200235, China

Received date: 2020-11-11

Revised date: 2020-11-30

Online published: 2021-02-24

Supported by

National Natural Science Foundation of China (51905091); Fundamental Research Funds for the Central Universities and Graduate Student Innovation Fund of Donghua University (CUSF-DH-D-2020053)

Fold

摘要

激光焊接在航空领域具有广泛的应用场景，基于视觉的激光焊接缺陷识别对于产品质量的提高至关重要。针对当前基于深度学习的激光焊接缺陷识别方法存在可解释性差的问题，提出了一种融合多尺度特征的类激活映射（MSF-CAM）方法。在训练阶段，以VGG16为骨架模型并将监督信息施加于多个尺度以促进模型对多尺度特征的学习。在测试阶段，对输出类别在多个尺度上的激活图进行叠加，并以此作为模型的判断依据。多尺度特征的融入不但增强了模型的可解释性，而且还提高了激光焊接缺陷识别的准确性。试验结果表明：MSF-CAM在测试集上的准确率为98.12%，识别单幅图像耗时8.28 ms。此外，MSF-CAM可以从边缘、轮廓这种初级特征的角度对模型的决策依据提供人类更容易理解的解释。

关键词： 激光焊接; 缺陷识别; 深度学习; 可解释性; 多尺度

本文引用格式

刘天元 , 郑杭彬 , 杨长祺 , 鲍劲松 , 汪俊亮 , 顾俊 . 面向激光焊接缺陷识别的可解释性深度学习方法[J]. 航空学报, 2022 , 43(4) : 524961 -524961 . DOI: 10.7527/S1000-6893.2021.24961

Abstract

Laser welding has a wide range of applications in the aerospace industry. Vision-based laser welding defect detection is crucial to the improvement of product quality. To overcome the problem of poor explainability in current deep learning-based laser welding defect detection methods, a method of Class Activation Mapping by Incorporating Multiscale Features (MSF-CAM) is proposed. In the training phase, VGG16 is used as the backbone network, and supervisory information is applied to multiple scales to facilitate learning of multiscale features. In the testing phase, the activation maps of the output categories on multiple scales are superimposed and used as the basis for the judgment by the model. Integration of multiscale features not only enhances explainability of the model, but also improves accuracy of laser welding defect detection. The test results show that the accuracy of MSF-CAM on the test set is 98.12% and the time used by the method to identify a single image is 8.28 ms. In addition, MSF-CAM can provide more understandable explanation for the decision-making basis of the model from the perspective of such primary features as edge and contour.

Key words： laser welding; defect recognition; deep learning; explainability; multiscale

参考文献

[1] 王连庆, 胡雅楠, 车志刚, 等. 激光冲击强化7075铝合金熔焊接头的疲劳性能[J]. 航空学报, 2021, 42(5):524320. WANG L Q, HU Y N, CHE Z G, et al. Fatigue performance of laser shock processed fusion welded 7075 Al alloy[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(5):524320(in Chinese).
[2] MENG G, JI B, HAN H, et al. Design and simulation of an innovative cylinder fabricated by selective laser melting[J]. Chinese Journal of Aeronautics, 2019, 32(1):133-142.
[3] 金俊龙, 郭德伦, 刘琦, 等. 激光成形TC17钛合金线性摩擦焊接头组织与力学性能[J]. 焊接学报, 2019, 40(6):126-130, 166. JIN J L, GUO D L, LIU Q, et al. Microstructure and mechanical properties of linear friction welding joint of TC17 titanium alloy fabricated by laser forming[J]. Transactions of the China Welding Institution, 2019, 40(6):126-130, 166(in Chinese).
[4] 刘浩东, 胡芳友, 戴京涛, 等. 军机薄壁结构损伤超声跨态处理新工艺[J]. 焊接学报, 2016, 37(10):77-80, 133. LIU H D, HU F Y, DAI J T, et al. New technology of ultrasonic processing across different phases in laser welding for damage repairing of thin-walled structure in military aircraft[J]. Transactions of the China Welding Institution, 2016, 37(10):77-80, 133(in Chinese).
[5] ABIOYE T E, ZUHAILAWATI H, AIZAD S, et al. Geometrical, microstructural and mechanical characterization of pulse laser welded thin sheet 5052-H32 aluminium alloy for aerospace applications[J]. Transactions of Nonferrous Metals Society of China, 2019, 29(4):667-679.
[6] 夏令, 吴友发, 余海松, 等. 铝锂合金激光焊接接头组织与缺陷研究进展[J]. 航空制造技术, 2018, 61(S2):77-81, 86. XIA L, WU Y F, YU H S, et al. Research progress on microstructure and defects in Al-Li alloy laser welded joint[J]. Aeronautical Manufacturing Technology, 2018, 61(2):77-81, 86(in Chinese).
[7] 宫建锋, 李慧知, 李俐群, 等. 基于同轴图像传感的激光焊接过程质量监测技术[J]. 焊接学报, 2019, 40(1):37-42, 162. GONG J F, LI H Z, LI L Q, et al. Quality monitoring technology of laser welding process based on coaxial image sensing[J]. Transactions of the China Welding Institution, 2019, 40(1):37-42, 162(in Chinese).
[8] 贺钧婷, 陈宝华, 孙文卿, 等. 基于机器视觉的激光辅助焊接[J]. 激光杂志, 2018, 39(11):47-50. HE J T, CHEN B H, SUN W Q, et al. Laser assisted welding based on machine-vision[J]. Laser Journal, 2018, 39(11):47-50(in Chinese).
[9] LUO M, SHIN Y C. Vision-based weld pool boundary extraction and width measurement during keyhole fiber laser welding[J]. Optics and Lasers in Engineering, 2015, 64:59-70.
[10] YOU D Y, GAO X D, KATAYAMA S. Monitoring of high-power laser welding using high-speed photographing and image processing[J]. Mechanical Systems and Signal Processing, 2014, 49(1-2):39-52.
[11] TANG X, ZHONG P, ZHANG L L, et al. A new method to assess fiber laser welding quality of stainless steel 304 based on machine vision and hidden Markov models[J]. IEEE Access, 2020, 8:130633-130646.
[12] GVNTHER J, PILARSKI P M, HELFRICH G, et al. Intelligent laser welding through representation, prediction, and control learning:An architecture with deep neural networks and reinforcement learning[J]. Mechatronics, 2016, 34:1-11.
[13] ZHANG Z H, LI B, ZHANG W F, et al. Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks[J]. Journal of Manufacturing Systems, 2020, 54:348-360.
[14] ZHANG Y X, YOU D Y, GAO X D, et al. Online monitoring of welding status based on a DBN model during laser welding[J]. Engineering, 2019, 5(4):671-678.
[15] YANG Y T, YANG R Z, PAN L H, et al. A lightweight deep learning algorithm for inspection of laser welding defects on safety vent of power battery[J]. Computers in Industry, 2020, 123:103306.
[16] WANG Q Y, JIAO W H, WANG P, et al. A tutorial on deep learning-based data analytics in manufacturing through a welding case study[J]. Journal of Manufacturing Processes, 2021, 63:2-13.
[17] CAI W, WANG J Z, JIANG P, et al. Application of sensing techniques and artificial intelligence-based methods to laser welding real-time monitoring:A critical review of recent literature[J]. Journal of Manufacturing Systems, 2020, 57:1-18.
[18] WANG B C, HU S J, SUN L, et al. Intelligent welding system technologies:State-of-the-art review and perspectives[J]. Journal of Manufacturing Systems, 2020, 56:373-391.
[19] 纪守领, 李进锋, 杜天宇, 等. 机器学习模型可解释性方法、应用与安全研究综述[J]. 计算机研究与发展, 2019, 56(10):2071-2096. JI S L, LI J F, DU T Y, et al. Survey on techniques, applications and security of machine learning interpretability[J]. Journal of Computer Research and Development, 2019, 56(10):2071-2096(in Chinese).
[20] SUNG A H. Ranking importance of input parameters of neural networks[J]. Expert Systems With Applications, 1998, 15(3-4):405-411.
[21] RIBEIRO M T, SINGH S, GUESTRIN C. "Why should I trust You? ":Explaining the predictions of any classifier[C]//KDD'16:Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016:1135-1144.
[22] SIMONYAN K, VEDALDI A, ZISSERMAN A. Deep inside convolutional networks:Visualising image classification models and saliency maps[DB/OL]. arXiv preprint:1312.6034, 2013.
[23] ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016:2921-2929.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv preprint:1409.1556, 2014.
[25] DENG J, DONG W, SOCHER R, et al. ImageNet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2009:248-255.

Options

文章导航

模态框（Modal）标题

摘要

本文引用格式

Abstract

参考文献