基于深度学习的结冰区域和厚度检测方法

doi:10.7527/S1000-6893.2023.29283

Abstract

Abstract:

Aircraft icing has become one of the biggest threats to safe flight of aircraft， accounting for more than half of military and civil aircraft accidents in the world. Therefore， it is extremely important to detect the icing conditions during flight. Traditional icing sensors can only detect icing information at fixed points and cannot provide feedback on the icing status of overall area. In this paper， a large-area icing detection method is proposed， and a real-time airfoil icing detection model is established by using deep learning algorithm. Airfoil icing area is detected and segmented in real time using the fuselage image captured by cameras as input， and the icing thickness of the airfoil leading edge is calculated. The experimental results from the self-built icing dataset show that the average accuracy of the algorithm is 99.32%， the mean intersection over union is 93.20%， the theoretical spatial resolution is approximately 0.4 mm at a distance of 1.5 m， the actual measurement error is within 1 mm， and the real-time detection frame rate is 4 frame/s. The experiments demonstrate that this method has excellent detection performance， and potentially lay research foundation for aircraft icing detection and ground icing wind tunnel tests.

Key words: deep learning, semantic segmentation, ice detection, thickness detection, UNet

CLC Number:

V244.1

Xin SU, Runcheng GUAN, Qiao WANG, Weizheng YUAN, Xianglian LYU, Yang HE. Ice area and thickness detection method based on deep learning[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 729283-729283.

Figures/Tables 11

Fig.1

Fig.2

Table 1

Fig.3

Table 2

Fig.4

Table 3

Fig.5

Fig.6

Fig.7

Table 4

References 27

1	桂业伟，周志宏，李颖晖，等. 关于飞机结冰的多重安全边界问题［J］. 航空学报， 2017， 38（2）： 520734.
	GUI Y W， ZHOU Z H， LI Y H， et al. Multiple safety boundaries protection on aircraft icing［J］. Acta Aeronautica et Astronautica Sinica， 2017， 38（2）： 520734 （in Chinese）.
2	朱东宇，张付昆，裴如男，等. 临界冰形确定方法及其对气动特性影响研究［J］. 空气动力学学报， 2016， 34（6）： 714-720.
	ZHU D Y， ZHANG F K， PEI R N， et al. Research on critical ice shape determination and its effects on aerodynamics［J］. Acta Aerodynamica Sinica， 2016， 34（6）： 714-720 （in Chinese）.
3	薛源，徐浩军，裴彬彬，等. 基于蒙特卡罗法的飞机结冰后动力学特性分析［J］. 实验流体力学， 2016， 30（2）： 26-31， 37.
	XUE Y， XU H J， PEI B B， et al. Dynamic characteristics analysis after aircraft icing based on Monte Carlo method［J］. Journal of Experiments in Fluid Mechanics， 2016， 30（2）： 26-31， 37 （in Chinese）.
4	YAN Z X， HE Y， YUAN W Z. Anti-icing skin with micro-nano structure inspired by Fargesia qinlingensis［J］. Transactions of Nanjing University of Aeronautics and Astronautics， 2023， 40（2）： 115-123.
5	张杰，周磊，张洪，等. 飞机结冰探测技术［J］. 仪器仪表学报， 2006， 27（12）： 1578-1586.
	ZHANG J， ZHOU L， ZHANG H， et al. Aircraft icing detection technology［J］. Chinese Journal of Scientific Instrument， 2006， 27（12）： 1578-1586 （in Chinese）.
6	任宏宇. 双余度结冰探测系统的研究与开发［D］. 武汉：华中科技大学， 2021： 4-10.
	REN H Y. Research and development of dual redundancy ice detection system［D］. Wuhan： Huazhong University of Science and Technology， 2021： 4-10 （in Chinese）.
7	ZHI X， CHO H C， WANG B， et al. Development of a capacitive ice sensor to measure ice growth in real time［J］. Sensors， 2015， 15（3）： 6688-6698.
8	FLATSCHER M， NEUMAYER M， BRETTERKLIE⁃ BER T. Maintaining critical infrastructure under cold climate conditions： A versatile sensing and heating concept［J］. Sensors and Actuators A： Physical， 2017， 267： 538-546.
9	王岩，王渊，朱程香，等. 谐振式飞机结冰探测传感器仿真及实验研究［J］. 南京航空航天大学学报， 2022， 54（2）： 267-273.
	WANG Y， WANG Y， ZHU C X， et al. Simulation and experimental study of aircraft icing detection sensor by resonant［J］. Journal of Nanjing University of Aeronautics & Astronautics， 2022， 54（2）： 267-273 （in Chinese）.
10	ZOU J H， YE L， GE J F. Ice type detection using an oblique end-face fibre-optic technique［J］. Measurement Science and Technology， 2013， 24（3）： 035201.
11	白天，朱春玲，李清英，等. 压电双晶片悬臂梁结构用于结冰探测的研究［J］. 航空学报， 2013， 34（5）： 1073-1082.
	BAI T， ZHU C L， LI Q Y， et al. Study of bimorph piezoelectric cantilever structure used on icing detection［J］. Acta Aeronautica et Astronautica Sinica， 2013， 34（5）： 1073-1082 （in Chinese）.
12	周志炜，李起栋，张鹏. 振管式结冰检测传感器设计优化及改进［J］. 传感器世界， 2017， 23（7）： 17-20.
	ZHOU Z W， LI Q D， ZHANG P. Optimization and improvement in design of vibrating tube icing detection sensor［J］. Sensor World， 2017， 23（7）： 17-20 （in Chinese）.
13	赵伟伟. 基于压电材料的飞机结冰探测系统［D］. 南京：南京航空航天大学， 2018： 24-40.
	ZHAO W W. Aircraft icing detection system based on piezoelectric materials［D］. Nanjing： Nanjing University of Aeronautics and Astronautics， 2018： 24-40 （in Chinese）.
14	赵伟伟，朱春玲，陶明杰，等. 超声导波技术用于飞机结冰探测的实验研究［J］. 压电与声光， 2018， 40（2）： 269-275.
	ZHAO W W， ZHU C L， TAO M J， et al. Experimental study on ultrasonic guided wave technology for aircraft icing detection［J］. Piezoelectrics & Acoustooptics， 2018， 40（2）： 269-275 （in Chinese）.
15	郭奕兵. 基于FBG传感和石墨烯薄膜的风机结冰监测与除冰［D］. 哈尔滨：哈尔滨工业大学， 2018： 16-36.
	GUO Y B. Graphene film combined with FBG sensor for icing monitoring and de-icing of wind turbine ［D］.Harbin： Harbin Institute of Technology， 2018： 16-36 （in Chinese）.
16	武晋文，王华征，宋继红. 振筒式结冰探测器硬件电路设计［J］. 集成电路应用， 2020， 37（7）： 20-21.
	WU J W， WANG H Z， SONG J H. Design of hardware circuit of ice detector with vibrating cylinder［J］. Application of IC， 2020， 37（7）： 20-21 （in Chinese）.
17	易贤，李维浩，王应宇，等. 飞机结冰传感器安装位置确定方法［J］. 实验流体力学， 2018， 32（2）： 48-54.
	YI X， LI W H， WANG Y Y， et al. Method of determining the location for aircraft icing prober［J］. Journal of Experiments in Fluid Mechanics， 2018， 32（2）： 48-54 （in Chinese）.
18	AKHLOUFI M， BENMESBAH N. Outdoor ice accretion estimation of wind turbine blades using computer vision［C］∥2014 Canadian Conference on Computer and Robot Vision. Piscataway： IEEE Press， 2014： 246-253.
19	ZHUGE J C， YU Z J， GAO J S， et al. Influence of color coatings on aircraft surface ice detection based on multi-wavelength imaging［J］. Optoelectronics Letters， 2016， 12（2）： 144-147.
20	郭龙，秦三春，李强，等. 基于机器视觉的飞机模型冰形轮廓提取方法研究［J］. 自动化与仪器仪表， 2020（6）： 15-20.
	GUO L， QIN S C， LI Q， et al. Research on ice contour extraction of aircraft model based on machine vision［J］. Automation & Instrumentation， 2020（6）： 15-20 （in Chinese）.
21	NOLTE M， KISTER N， MAURER M. Assessment of deep convolutional neural networks for road surface classification［C］∥2018 21st International Conference on Intelligent Transportation Systems （ITSC）. Piscataway： IEEE Press， 2018： 381-386.
22	LIANG C J， GE J F， ZHANG W， et al. Winter road surface status recognition using deep semantic segmentation network［C］∥Proceedings of the International Workshop on Atmospheric Icing of Structures （IWAIS 2019）. Cham： Springer， 2020： 91-100.
23	ZEILER M D， FERGUS R. Visualizing and understanding convolutional networks［C］∥European Conference on Computer Vision. Cham： Springer， 2014： 818-833.
24	RONNEBERGER O， FISCHER P， BROX T. U-net： Convolutional networks for biomedical image segmentation［C］∥International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham： Springer， 2015： 234-241.
25	JIA Y Q， SHELHAMER E， DONAHUE J， et al. Caffe： Convolutional architecture for fast feature embedding［C］∥Proceedings of the 22nd ACM international conference on Multimedia. New York： ACM， 2014： 675-678.
26	HE K M， ZHANG X Y， REN S Q， et al. Delving deep into rectifiers： Surpassing human-level performance on ImageNet classification［C］∥2015 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2016： 1026-1034.
27	ZHANG Z. A flexible new technique for camera calibration［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2000， 22（11）： 1330-1334.

Batch size	1	2	4	8	16
平均准确率/%	97.34	97.96	98.02	99.36	99.32
平均交并比/%	80.57	84.44	85.04	93.61	93.20

类别	准确率/%	召回率/%	交并比/%
背景	99.61	99.67	99.28
冰	99.68	92.57	87.12

编号	计算耗时/s	总耗时/s
平均	0.57	1.88
1	0.55	1.80
2	0.62	2.10
3	0.59	1.90
4	0.48	1.80
5	0.61	1.80
6	0.54	1.90
7	0.58	2.00
8	0.60	1.80
9	0.56	1.80

编号	1	2	3	4
手动测量/mm	9.5±0.5	8.0±0.5	10.0±0.5	10.5±0.5
视觉检测/mm	9.6±0.4	8.0±0.4	10.0±0.4	10.4±0.4

[1]	Jiaqi LIU, Rongqian CHEN, Jinhua LOU, Xu HAN, Hao WU, Yancheng YOU. Aerodynamic shape optimization of high-speed helicopter rotor airfoil based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(9): 529828-529828.
[2]	Xudong LUO, Yiquan WU, Jinlin CHEN. Research progress on deep learning methods for object detection and semantic segmentation in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 28822-028822.
[3]	Haiqiao LIU, Meng LIU, Zichao GONG, Jing DONG. A review of image matching methods based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(3): 28796-028796.
[4]	Guochi GAO, Bo ZHANG, Jingze QUAN, Chong YIN, Li DING, Yubiao JIANG. Airworthiness certification technology of normal aircraft natural icing flight test [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(1): 128531-128531.
[5]	Liqun CHEN, Xu ZOU, Lei ZHANG, Yingpan ZHU, Gang WANG, Jinyong CHEN. On⁃board intelligent target detection technology based on domestic commercial components [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 728860-728860.
[6]	Hongjian ZHANG, Yanxin ZHANG, Jianjun XIONG, Zhao ZHAO, Lin RAN, Xian YI. Numerical simulation of phased array ultrasonic beam propagation characteristics in ice layer [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 729289-729289.
[7]	Jinyi LIU, Jianjun XIONG, Kang GUI, Junfeng GE, Lin YE. Integrated ice detection technology based on multispectral and complex impedance principles [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 729309-729309.
[8]	Pengyu LIU, Xueyao ZHU. Semantic parsing technology of air traffic control instruction in fusion airspace based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S1): 727592-727592.
[9]	Lei HE, Weiqi QIAN, Kangsheng DONG, Xian YI, Congcong CHAI. Aerodynamic characteristics modeling of iced airfoil based on convolution neural networks [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(5): 126434-126434.
[10]	Peng DING, Yafei SONG. A cost-sensitive method for aerial target intention recognition [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(24): 328551-328551.
[11]	Xiaohang LI, Jianjiang ZHOU. Multi⁃scale modality fusion network based on adaptive memory length [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(22): 628977-628977.
[12]	Haowen LUO, Shaoming HE, Tianyu JIN, Zichao LIU. Impact-angle-constrained with time-minimum guidance algorithm based on transfer learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(19): 328400-328400.
[13]	Yunhe ZHAO, Shengnan WANG. Solution to stress intensity factor by weight function method based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(19): 228367-228367.
[14]	Shuyi GAO, Defu LIN, Duo ZHENG, Xinyu HU. Intelligent cooperative interception strategy of aircraft against cluster attack [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(18): 328301-328301.
[15]	Yubin YUAN, Yiquan WU, Langyue ZHAO, Jinlin CHEN, Qichang ZHAO. Research progress of UAV aerial video multi⁃object detection and tracking based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(18): 28334-028334.

Ice area and thickness detection method based on deep learning

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 27

Related Articles 15

Recommended Articles

Metrics

Comments