基于红外和可见光图像融合的铺丝缺陷检测方法

doi:10.7527/S1000-6893.2021.25187

材料工程与机械制造

本期目录 | 过刊浏览 | 高级检索

前一篇 | 后一篇

基于红外和可见光图像融合的铺丝缺陷检测方法

康硕¹, 柯臻铮², 王璇¹, 朱伟东¹

1. 浙江大学机械工程学院, 杭州 310027;
2. 浙江大学先进技术研究院, 杭州 310027

收稿日期:2020-12-31 修回日期:2021-01-11 出版日期:2022-03-15 发布日期:2021-02-18
通讯作者: 柯臻铮 E-mail:kzzcaen@zju.edu.cn
基金资助:
浙江省“尖兵”“领雁”研发攻关计划（2022C01134）

Detection method of defects in automatic fiber placement based on fusion of infrared and visible images

KANG Shuo¹, KE Zhenzheng², WANG Xuan¹, ZHU Weidong¹

1. College of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China;
2. Institute of Advanced Technology, Zhejiang University, Hangzhou 310027, China

Received:2020-12-31 Revised:2021-01-11 Online:2022-03-15 Published:2021-02-18
Supported by:
Pioneer and "Leading Goose" Research and Development Program of Zhejiang (2022C01134)

摘要/Abstract

摘要： 为提高纤维自动铺放（AFP）的质量，分析了目前视觉检测技术中单光谱检测技术成像的局限性，提出一种基于深度学习的红外与可见光联合检测缺陷手段以实现对铺放缺陷的检测、定位和分类。利用热红外图像与可见光图像中易检缺陷种类不同的特点，采用特征融合网络将两种光谱信息融合从而改善检测效果。考虑在线检测对实时性要求较高，为提高检测速度，采用单阶段检测网络作为检测框架，并依据纤维缺陷的长宽比分布严重不均的特点分析了单阶段检测网络中基于锚框方法的不足，提出采用无锚框的检测框架，增加改进的特征金字塔网络结构进行多尺度预测。以全类平均正确率（mAP）为衡量指标，实验结果相比单光谱检测方法、基于锚框检测方法和未使用改进特征金字塔结构分别提升了6.30%、6.64%和1.02%。通过Tensor RT加速后检测网络对每张608像素×608像素图像的检测时间小于20 ms，满足实时检测的需求，检测平均召回率超过88%，平均精确率超过82%，满足生产准确度需求。数据在试验台进行离线检测验证得出，并在大型龙门铺丝机上进行了在线测试。

关键词: 纤维自动铺放, 缺陷检测, 深度学习, 机器视觉, 多光谱融合

Abstract: To improve the quality of Automatic Fiber Placement (AFP), this paper analyzes the limitations of single spectrum detection technology in the field of visual inspection, and proposes a method based on deep learning for defect detection by fusion of infrared and visible images to realize detection, location and classification of fiber placement defects.According to the difference between the defects in thermal infrared image and visible image, the feature fusion network is used to fuse the two kinds of spectral information to improve the detection effect.To improve the detection speed, the single-stage detection network is used as the detection framework.We analyze the shortcomings of anchor-based method in the single-stage detection network according to the characteristics of seriously uneven distribution of length-width ratio of fiber defects, and propose a detection method with anchor-free network, in which an improved feature pyramid network structure is added for multi-scale prediction.Using mean Average Precision (mAP) as the measurement index, the experimental results improved by 6.30%, 6.64% and 1.02% respectively compared with single spectrum detection method, anchor-based detection method and the detection method without improved feature pyramid structure.The detection time of each 608 pixels×608 pixels image is less than 20 ms after acceleration by Tensor RT, which meets the demand of real-time detection.The average recall of detection is more than 88%, and the average precision is more than 82%, which meets the demand of production accuracy.The data of this paper is verified by off-line detection on the test-bed, and is tested online on large gantry AFP equipment.

Key words: automatic fiber placement, defects detection, deep learning, machine vision, multi-spectrum fusion

中图分类号:

康硕, 柯臻铮, 王璇, 朱伟东. 基于红外和可见光图像融合的铺丝缺陷检测方法[J]. 航空学报, 2022, 43(3): 425187-425187.

KANG Shuo, KE Zhenzheng, WANG Xuan, ZHU Weidong. Detection method of defects in automatic fiber placement based on fusion of infrared and visible images[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(3): 425187-425187.

参考文献

[1] CROFT K, LESSARD L, PASINI D, et al.Experimental study of the effect of automated fiber placement induced defects on performance of composite laminates[J].Composites Part A:Applied Science and Manufacturing, 2011, 42(5):484-491.
[2] HARIK R, SAIDY C, WILLIAMS S J, et al.Automated fiber placement defect identity cards:Cause, anticipation, existence, significance, and progression[C]//SAMPE 18.Covina:SAMPE, 2018.
[3] 袁鑫超.碳纤维复合材料多层内部结构及缺陷检测方法研究[D].成都:电子科技大学, 2018. YUAN X C.Study on multi-layer layer fiber structure and defect detection of carbon fiber composites[D].Chengdu:University of Electronic Science and Technology of China, 2018(in Chinese).
[4] CEMENSKA J, RUDBERG T, HENSCHEID M, et al.AFP automated inspection system performance and expectations[C]//Aerotech Congress & Exhibition.Warrendale:SAE International, 2017:2150.
[5] RITTER J A, SJOGREN J F.Real-time infrared thermography inspection and control for automated composite marterial layup:US7513964[P].2009-04-07.
[6] DENKENA B, SCHMIDT C, VÖLTZER K, et al.Thermographic online monitoring system for Automated Fiber Placement processes[J].Composites Part B:Engineering, 2016, 97:239-243.
[7] SCHMIDT C, DENKENA B, VÖLTZER K, et al.Thermal image-based monitoring for the automated fiber placement process[J].Procedia CIRP, 2017, 62:27-32.
[8] GREGORY E D, JUAREZ P D.In-situ thermography of automated fiber placement parts[J].AIP Conference Proceedings, 2018, 1949(1):060005.
[9] 黄松岭, 李路明, 杨海青, 等.复合材料胶接缺陷的红外热像检测[J].宇航材料工艺, 2002, 32(6):43-46. HUANG S L, LI L M, YANG H Q, et al.Evaluation of composites bonding defects by infrared imaging testing[J].Aerospace Materials & Technology, 2002, 32(6):43-46(in Chinese).
[10] 文立伟, 宋清华, 秦丽华, 等.基于机器视觉与UMAC的自动铺丝成型构件缺陷检测闭环控制系统[J].航空学报, 2015, 36(12):3991-4000. WEN L W, SONG Q H, QIN L H, et al.Defect detection and closed-loop control system for automated fiber placement forming components based on machine vision and UMAC[J].Acta Aeronautica et Astronautica Sinica, 2015, 36(12):3991-4000(in Chinese).
[11] ZAMBAL S, HEINDL C, EITZINGER C, et al.End-to-end defect detection in automated fiber placement based on artificially generated data[C]//Proc SPIE 11172, Fourteenth International Conference on Quality Control by Artificial Vision.Lausanne:International Society for Optics and Photonics, 2019:11172.
[12] 路浩, 陈原.基于机器视觉的碳纤维预浸料表面缺陷检测方法[J].纺织学报, 2020, 41(4):51-57. LU H, CHEN Y.Surface defect detection method of carbon fiber prepreg based on machine vision[J].Journal of Textile Research, 2020, 41(4):51-57(in Chinese).
[13] CHEN M J, JIANG M, LIU X L, et al.Intelligent inspection system based on infrared vision for automated fiber placement[C]//2018 IEEE International Conference on Mechatronics and Automation.Piscataway:IEEE Press, 2018:918-923.
[14] SACCO C, BAZ RADWAN A, ANDERSON A, et al.Machine learning in composites manufacturing:A case study of Automated Fiber Placement inspection[J].Composite Structures, 2020, 250:112514.
[15] SACCO C.Machine learning methods for rapid inspection of automated fiber placement manufactured composite structures[D].Columbia:University of South Carolina, 2019.
[16] 蔡志强, 肖军, 文立伟, 等.基于预浸纱自动铺放缺陷的分割算法[J].航空材料学报, 2017, 37(2):21-27. CAI Z Q, XIAO J, WEN L W, et al.Algorithm of defect segmentation for AFP based on prepregs[J].Journal of Aeronautical Materials, 2017, 37(2):21-27(in Chinese).
[17] REN S Q, HE K M, GIRSHICK R, et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[18] GIRSHICK R, DONAHUE J, DARRELL T, et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2014:580-587.
[19] LAW H, DENG J.CornerNet:Detecting objects as paired keypoints[J].International Journal of Computer Vision, 2020, 128(3):642-656.
[20] FLIR.Free FLIR thermal dataset for algorithm training[DB/OL].(2020-06-21)[2020-06-21].https://www.flir.com/oem/adas/adas-dataset-form.
[21] DENG J, DONG W, SOCHER R, et al.ImageNet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2009:248-255.
[22] WOLPERT A, TEUTSCH M, SARFRAZ M S, et al.Anchor-free small-scale multispectral pedestrian detection[DB/OL].arXiv preprint:2008.08418, 2020.
[23] LIN T Y, DOLLÁR P, GIRSHICK R, et al.Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2017:936-944.
[24] LIU S, QI L, QIN H F, et al.Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press, 2018:8759-8768.
[25] ZHANG Z.A flexible new technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.

E-mail：hkxb@buaa.edu.cn

关于我们

期刊社服务

专业学科

封面文章

友情链接

主管单位：中国科学技术协会主办单位：中国航空学会北京航空航天大学

基于红外和可见光图像融合的铺丝缺陷检测方法

Detection method of defects in automatic fiber placement based on fusion of infrared and visible images

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	刘欢, 刘骁佳, 王宇斐, 王宁, 曹立俊. 基于复合卷积层神经网络结构的焊缝缺陷分类技术[J]. 航空学报, 2022, 43(S1): 726928-726928.
[2]	苏翎菲, 化永朝, 董希旺, 任章. 人与无人机集群多模态智能交互方法[J]. 航空学报, 2022, 43(S1): 727001-727001.
[3]	赵良玉, 李丹, 赵辰悦, 蒋飞. 无人机自主降落标识检测方法若干研究进展[J]. 航空学报, 2022, 43(9): 25882-025882.
[4]	曹明, 王鹏, 左洪福, 曾海军, 孙见忠, 杨卫东, 魏芳, 陈雪峰. 民用航空发动机故障诊断与健康管理现状、挑战与机遇Ⅱ: 地面综合诊断、寿命管理和智能维护维修决策[J]. 航空学报, 2022, 43(9): 625574-625574.
[5]	刘佳奇, 冯蕴雯, 路成, 薛小锋, 潘维煌. 基于智能神经网络的航空发动机运行安全分析[J]. 航空学报, 2022, 43(9): 625375-625375.
[6]	邵怡韦, 陈嘉宇, 林翠颖, 万程, 葛红娟, 石智龙. 小训练样本下齿轮箱故障诊断:一种基于改进深度森林的方法[J]. 航空学报, 2022, 43(8): 625429-625429.
[7]	刘子超, 王江, 何绍溟, 李宇飞. 基于预测校正的落角约束计算制导方法[J]. 航空学报, 2022, 43(8): 325433-325433.
[8]	孙灿飞, 黄林然, 沈勇. 复杂工况下的直升机行星传动轮系故障诊断[J]. 航空学报, 2022, 43(8): 625480-625480.
[9]	康玉祥, 陈果, 尉询楷, 周磊. 深度残差对冲网络及其在滚动轴承故障诊断中的应用[J]. 航空学报, 2022, 43(8): 625201-625201.
[10]	张洲宇, 曹云峰, 范彦铭. 低空小型无人机空域冲突视觉感知技术研究进展[J]. 航空学报, 2022, 43(8): 25645-025645.
[11]	刘占, 张俊, 尹佳, 赵万华. 基于机器视觉的立铣刀几何与状态参数在机检测[J]. 航空学报, 2022, 43(7): 425593-425593.
[12]	刘芳, 孙亚楠. 基于自适应融合网络的无人机目标跟踪算法[J]. 航空学报, 2022, 43(7): 325522-325522.
[13]	吕帅帅, 杨宇, 王彬文, 殷晨飞. 基于改进FaceNet的飞行器结构裂纹识别方法[J]. 航空学报, 2022, 43(6): 525463-525463.
[14]	刘天元, 郑杭彬, 杨长祺, 鲍劲松, 汪俊亮, 顾俊. 面向激光焊接缺陷识别的可解释性深度学习方法[J]. 航空学报, 2022, 43(4): 524961-524961.
[15]	战庆亮, 白春锦, 张宁, 葛耀君. 基于时程卷积自编码的机翼绕流特征识别方法[J]. 航空学报, 2022, 43(11): 526531-526531.