基于红外和可见光图像融合的铺丝缺陷检测方法

doi:10.7527/S1000-6893.2021.25187

Abstract

Abstract: To improve the quality of Automatic Fiber Placement (AFP), this paper analyzes the limitations of single spectrum detection technology in the field of visual inspection, and proposes a method based on deep learning for defect detection by fusion of infrared and visible images to realize detection, location and classification of fiber placement defects.According to the difference between the defects in thermal infrared image and visible image, the feature fusion network is used to fuse the two kinds of spectral information to improve the detection effect.To improve the detection speed, the single-stage detection network is used as the detection framework.We analyze the shortcomings of anchor-based method in the single-stage detection network according to the characteristics of seriously uneven distribution of length-width ratio of fiber defects, and propose a detection method with anchor-free network, in which an improved feature pyramid network structure is added for multi-scale prediction.Using mean Average Precision (mAP) as the measurement index, the experimental results improved by 6.30%, 6.64% and 1.02% respectively compared with single spectrum detection method, anchor-based detection method and the detection method without improved feature pyramid structure.The detection time of each 608 pixels×608 pixels image is less than 20 ms after acceleration by Tensor RT, which meets the demand of real-time detection.The average recall of detection is more than 88%, and the average precision is more than 82%, which meets the demand of production accuracy.The data of this paper is verified by off-line detection on the test-bed, and is tested online on large gantry AFP equipment.

Key words: automatic fiber placement, defects detection, deep learning, machine vision, multi-spectrum fusion

CLC Number:

KANG Shuo, KE Zhenzheng, WANG Xuan, ZHU Weidong. Detection method of defects in automatic fiber placement based on fusion of infrared and visible images[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(3): 425187-425187.

References

[1] CROFT K, LESSARD L, PASINI D, et al.Experimental study of the effect of automated fiber placement induced defects on performance of composite laminates[J].Composites Part A:Applied Science and Manufacturing, 2011, 42(5):484-491.
[2] HARIK R, SAIDY C, WILLIAMS S J, et al.Automated fiber placement defect identity cards:Cause, anticipation, existence, significance, and progression[C]//SAMPE 18.Covina:SAMPE, 2018.
[3] 袁鑫超.碳纤维复合材料多层内部结构及缺陷检测方法研究[D].成都:电子科技大学, 2018. YUAN X C.Study on multi-layer layer fiber structure and defect detection of carbon fiber composites[D].Chengdu:University of Electronic Science and Technology of China, 2018(in Chinese).
[4] CEMENSKA J, RUDBERG T, HENSCHEID M, et al.AFP automated inspection system performance and expectations[C]//Aerotech Congress & Exhibition.Warrendale:SAE International, 2017:2150.
[5] RITTER J A, SJOGREN J F.Real-time infrared thermography inspection and control for automated composite marterial layup:US7513964[P].2009-04-07.
[6] DENKENA B, SCHMIDT C, VÖLTZER K, et al.Thermographic online monitoring system for Automated Fiber Placement processes[J].Composites Part B:Engineering, 2016, 97:239-243.
[7] SCHMIDT C, DENKENA B, VÖLTZER K, et al.Thermal image-based monitoring for the automated fiber placement process[J].Procedia CIRP, 2017, 62:27-32.
[8] GREGORY E D, JUAREZ P D.In-situ thermography of automated fiber placement parts[J].AIP Conference Proceedings, 2018, 1949(1):060005.
[9] 黄松岭, 李路明, 杨海青, 等.复合材料胶接缺陷的红外热像检测[J].宇航材料工艺, 2002, 32(6):43-46. HUANG S L, LI L M, YANG H Q, et al.Evaluation of composites bonding defects by infrared imaging testing[J].Aerospace Materials & Technology, 2002, 32(6):43-46(in Chinese).
[10] 文立伟, 宋清华, 秦丽华, 等.基于机器视觉与UMAC的自动铺丝成型构件缺陷检测闭环控制系统[J].航空学报, 2015, 36(12):3991-4000. WEN L W, SONG Q H, QIN L H, et al.Defect detection and closed-loop control system for automated fiber placement forming components based on machine vision and UMAC[J].Acta Aeronautica et Astronautica Sinica, 2015, 36(12):3991-4000(in Chinese).
[11] ZAMBAL S, HEINDL C, EITZINGER C, et al.End-to-end defect detection in automated fiber placement based on artificially generated data[C]//Proc SPIE 11172, Fourteenth International Conference on Quality Control by Artificial Vision.Lausanne:International Society for Optics and Photonics, 2019:11172.
[12] 路浩, 陈原.基于机器视觉的碳纤维预浸料表面缺陷检测方法[J].纺织学报, 2020, 41(4):51-57. LU H, CHEN Y.Surface defect detection method of carbon fiber prepreg based on machine vision[J].Journal of Textile Research, 2020, 41(4):51-57(in Chinese).
[13] CHEN M J, JIANG M, LIU X L, et al.Intelligent inspection system based on infrared vision for automated fiber placement[C]//2018 IEEE International Conference on Mechatronics and Automation.Piscataway:IEEE Press, 2018:918-923.
[14] SACCO C, BAZ RADWAN A, ANDERSON A, et al.Machine learning in composites manufacturing:A case study of Automated Fiber Placement inspection[J].Composite Structures, 2020, 250:112514.
[15] SACCO C.Machine learning methods for rapid inspection of automated fiber placement manufactured composite structures[D].Columbia:University of South Carolina, 2019.
[16] 蔡志强, 肖军, 文立伟, 等.基于预浸纱自动铺放缺陷的分割算法[J].航空材料学报, 2017, 37(2):21-27. CAI Z Q, XIAO J, WEN L W, et al.Algorithm of defect segmentation for AFP based on prepregs[J].Journal of Aeronautical Materials, 2017, 37(2):21-27(in Chinese).
[17] REN S Q, HE K M, GIRSHICK R, et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[18] GIRSHICK R, DONAHUE J, DARRELL T, et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2014:580-587.
[19] LAW H, DENG J.CornerNet:Detecting objects as paired keypoints[J].International Journal of Computer Vision, 2020, 128(3):642-656.
[20] FLIR.Free FLIR thermal dataset for algorithm training[DB/OL].(2020-06-21)[2020-06-21].https://www.flir.com/oem/adas/adas-dataset-form.
[21] DENG J, DONG W, SOCHER R, et al.ImageNet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2009:248-255.
[22] WOLPERT A, TEUTSCH M, SARFRAZ M S, et al.Anchor-free small-scale multispectral pedestrian detection[DB/OL].arXiv preprint:2008.08418, 2020.
[23] LIN T Y, DOLLÁR P, GIRSHICK R, et al.Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2017:936-944.
[24] LIU S, QI L, QIN H F, et al.Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2018:8759-8768.
[25] ZHANG Z.A flexible new technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.

Detection method of defects in automatic fiber placement based on fusion of infrared and visible images

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	SU Lingfei, HUA Yongzhao, DONG Xiwang, REN Zhang. Human-UAV swarm multi-modal intelligent interaction methods [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 727001-727001.
[2]	CAO Ming, WANG Peng, ZUO Hongfu, ZENG Haijun, SUN Jianzhong, YANG Weidong, WEI Fang, CHEN Xuefeng. Current status, challenges and opportunities of civil aero-engine diagnostics & health management Ⅱ: Comprehensive off-board diagnosis, life management and intelligent condition based MRO [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(9): 625574-625574.
[3]	LIU Jiaqi, FENG Yunwen, LU Cheng, XUE Xiaofeng, PAN Weihuang. Safety analysis of aero-engine operation based on intelligent neural network [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(9): 625375-625375.
[4]	ZHAO Liangyu, LI Dan, ZHAO Chenyue, JIANG Fei. Some achievements on detection methods of UAV autonomous landing markers [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(9): 25882-025882.
[5]	KANG Yuxiang, CHEN Guo, WEI Xunkai, ZHOU Lei. Deep residual hedging network and its application in fault diagnosis of rolling bearings [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 625201-625201.
[6]	ZHANG Zhouyu, CAO Yunfeng, FAN Yanming. Research progress of vision based aerospace conflict sensing technologies for small unmanned aerial vehicle in low altitude [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 25645-025645.
[7]	LIU Zichao, WANG Jiang, HE Shaoming, LI Yufei. A computational guidance algorithm for impact angle control based on predictor-corrector concept [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 325433-325433.
[8]	SUN Canfei, HUANG Linran, SHEN Yong. Fault diagnosis of helicopter planetary gear transmission system under complex operating conditions [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 625480-625480.
[9]	SHAO Yiwei, CHEN Jiayu, LIN Cuiying, WAN Cheng, GE Hongjuan, SHI Zhilong. Gearbox fault diagnosis with small training samples: An improved deep forest based method [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(8): 625429-625429.
[10]	LIU Zhan, ZHANG Jun, YIN Jia, ZHAO Wanhua. On-machine detection of geometric and state parameters of end mills based on machine vision [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(7): 425593-425593.
[11]	LIU Fang, SUN Yanan. UAV target tracking algorithm based on adaptive fusion network [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(7): 325522-325522.
[12]	LYU Shuaishuai, YANG Yu, WANG Binwen, YIN Chenfei. A crack identification method of aircraft structure based on improved FaceNet [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(6): 525463-525463.
[13]	LIU Tianyuan, ZHENG Hangbin, YANG Changqi, BAO Jinsong, WANG Junliang, GU Jun. Explainable deep learning method for laser welding defect detection [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(4): 524961-524961.
[14]	ZHAN Qingliang, BAI Chunjin, ZHANG Ning, GE Yaojun. Feature extraction method of flow around airfoil based on time-history convolutional autoencoder [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(11): 526531-526531.
[15]	YI Jianmiao, DENG Feng, QIN Ning, LIU Xueqiang. Fast prediction of transonic flow field using deep learning method [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(11): 526747-526747.