铁路外部环境无人机图像未知风险检测方法

doi:10.7527/S1000-6893.2024.31262

Abstract

Abstract:

Common hazards as well as unknown risks （including mudslides， rockfalls， animal intrusion， etc.） in the external environment of railroad seriously threaten the safe operation of railroads， requiring frequent time-consuming and laborious inspections by inspectors， but the scope of inspections is still very limited. At present， the low-altitude economy has become China’s new quality productivity representative， and the UAV has innate inspection advantages of high altitude， long distance， and small impact from the terrain and railroad maintenance windows. To overcome the challenge of sparse samples and random uncertainty of unknown risks in the external environment of the railroad， this paper utilizes UAVs for remote sensing image acquisition along the railroad， and proposes an unknown risk detection framework based on Faster R-CNN. Firstly， a novel targeted and multi-classification decoupling training strategy is designed and integrated in the unknown risk detection framework， which significantly improves the performance of general object detection and avoids misclassifying unknown risk objects as background. Secondly， the virtual feature synthesis method of VOS（Visual Object Segmentation） is improved， and similarity-based feature space sampling is designed to obtain a generalized unknown risk object feature representation by performing multivariate Gaussian distribution parameter estimation and resampling based on the construction of instance-level object feature space. Subsequently， an energy-based uncertainty measurement is utilized to measure the uncertainty of instance-level features， and losses are calculated accordingly to induce the network to optimize the decision boundaries for common and unknown risk categories. Finally， quantitative and qualitative experimental analyses are conducted on the collected railroad external environment dataset， open-source drone dataset， and generalization test data. The proposed method achieves 95.7% mAP₅₀ in common hazard identification， while achieving 98% and 80.8% Recall₅₀ in the test set and generalization test data， respectively. The experimental results show that the proposed method has high detection ability for unknown risk objects， while ensuring high recognition rate of common hazard categories.

Key words: UAV, remote sensing imagery, low-altitude economy, deep learning, external environment of railroad, unknown risk object detection

CLC Number:

V279

Fanteng MENG, Yong QIN, Jing CUI, Yunpeng WU, Zicheng ZHANG, Shaowei WEI. Unknown risk detection in external environment of railroad using UAV images[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531262.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Training parameter settings

超参数	参数/配置
输入图像尺寸	800×1 333
样本训练累计迭代总次数	18 000
训练前/后期迭代次数边界	10 000
Batch Size	6
Num Workers	8
初始学习率	0.01
学习率衰减策略	10倍线性衰减
学习率衰减的迭代次数	（12 000， 16 000）
预热迭代次数warmup	100
优化器	随机梯度下降（SGD）
实例级特征集合存储长度 $n$	1 000
单次重采样样本数量	1 000
$ϵ$ 取值	1%

Table 1

Table 2

Fig.5

Fig.6

Table 3

Fig.7

Fig.8

Table 4

Table 5

Ablation of objectness and muti-classification decoupling training strategy

训练策略

Recall₅₀/%

Recall_50~95/%

联合训练策略

79.3

（+0）

46.3

（+0）

联合训练策略+分类分支增加

L o b j

计算

78.9

（-0.4）

48.0

（+1.7）

解耦训练策略+分类分支增加

L o b j

计算

80.8

（+1.5）

48.9

（+2.6）

Table 5

References 37

[1]	毛天宇. 中国铁路运营里程［OL］. 北京：新京报， 2024. （2024-01-10）［2024-07-01］. .
	MAO T Y. China railway operating mileage［OL］. Beijing： New Beijing News， 2024. （2024-01-10）［2024-07-01］. （in Chinese）.
[2]	王薇. 京沪高铁线遭彩钢板撞击［OL］. 北京：北京青年报， 2018. （2018-08-14）［2024-07-01］. .
	WANG W. Beijing-Shanghai High speed rail line hit by color steel plate［OL］. Beijing： Beijing Youth Daily， 2018. （2018-08-14）［2024-07-01］. （in Chinese）.
[3]	晓明. 飘物侵袭接触网逼停动车［OL］. 泉州：泉州网， 2024. （2024-06-27）［2024-07-01］. .
	XIAO M. Floating objects invade the catenary and force the train to stop［OL］. Quanzhou： Quanzhou Network， 2024. （2024-06-27）［2024-07-01］. （in Chinese）.
[4]	刘楒睿. 泥石流致D2809次列车脱线［OL］. 长沙：极目新闻， 2022. （2022-06-04）［2024-07-01］. .
	LIU S X. Debris flow causing derailment of train D2809［OL］. Changsha： Jimu News， 2022. （2022-06-04）［2024-07-01］. （in Chinese）.
[5]	川黔铁路突发山体崩塌落石抢修［OL］. 北京：中国政府网， 2013. （2013-04-02）［2024-07-01］. .
	Sudden mountain collapse and rockfall on the Sichuan Guizhou Railway［OL］. Beijing： Chinese government website， 2013. （2013-04-02）［2024-07-01］. （in Chinese）.
[6]	WU Y P， CHEN P， QIN Y， et al. Automatic railroad track components inspection using hybrid deep learning framework［J］. IEEE Transactions on Instrumentation and Measurement， 2023， 72： 5011415.
[7]	WU Y P， QIN Y， QIAN Y， et al. Hybrid deep learning architecture for rail surface segmentation and surface defect detection［J］. Computer-Aided Civil and Infrastructure Engineering， 2022， 37（2）： 227-244.
[8]	CUI J， QIN Y， WU Y P， et al. Skip connection YOLO architecture for noise barrier defect detection using UAV-based images in high-speed railway［J］. IEEE Transactions on Intelligent Transportation Systems， 2023， 24（11）： 12180-12195.
[9]	WU Y P， QIN Y， QIAN Y， et al. Automatic detection of arbitrarily oriented fastener defect in high-speed railway［J］. Automation in Construction， 2021， 131： 103913.
[10]	WU Y P， MENG F T， QIN Y， et al. UAV imagery based potential safety hazard evaluation for high-speed railroad using Real-time instance segmentation［J］. Advanced Engineering Informatics， 2023， 55： 101819.
[11]	TONG L， WANG Z P， JIA L M， et al. Fully decoupled residual ConvNet for real-time railway scene parsing of UAV aerial images［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 23（9）： 14806-14819.
[12]	CHEN P， WU Y P， QIN Y， et al. All-in-one YOLO architecture for safety hazard detection of environment along high-speed railway［C］∥ 2022 Global Reliability and Prognostics and Health Management （PHM-Yantai）. Piscataway： IEEE Press， 2022： 1-7.
[13]	ZOHAR O， WANG K C， YEUNG S. PROB： Probabilistic objectness for open world object detection［EB/OL］. 2022： 2212.01424. .
[14]	DU X F， WANG Z N， CAI M， et al. VOS： Learning what you don’t know by virtual outlier synthesis［EB/OL］. 2022： 2202.01197. .
[15]	WU A M， DENG C. TIB： Detecting unknown objects via two-stream information bottleneck［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2024， 46（1）： 611-625.
[16]	TONG L， JIA L M， GENG Y X， et al. Anchor-adaptive railway track detection from unmanned aerial vehicle images［J］. Computer-Aided Civil and Infrastructure Engineering， 2023， 38（18）： 2666-2684.
[17]	MU Z H， QIN Y， YU C C， et al. Adaptive cropping shallow attention network for defect detection of bridge girder steel using unmanned aerial vehicle images［J］. Journal of Zhejiang University： Science A， 2023， 24（3）： 243-256.
[18]	GUDOVSKIY D， ISHIZAKA S， KOZUKA K. CFLOW-AD： Real-time unsupervised anomaly detection with localization via conditional normalizing flows［C］∥ 2022 IEEE/CVF Winter Conference on Applications of Computer Vision （WACV）. Piscataway： IEEE Press， 2022： 1819-1828.
[19]	YU J W， ZHENG Y， WANG X， et al. FastFlow： Unsupervised anomaly detection and localization via 2D normalizing flows［EB/OL］. 2021： 2111.07677. .
[20]	YOU Z Y， CUI L， SHEN Y J， et al. A unified model for multi-class anomaly detection［C］∥ 36th Conference on Neural Information Processing Systems （NeurIPS 2022）. New Orleans： NeurIPS Proceedings， 2022： 1-14.
[21]	BATZNER K， HECKLER L， KÖNIG R. EfficientAD： Accurate visual anomaly detection at millisecond-level latencies［C］∥ 2024 IEEE/CVF Winter Conference on Applications of Computer Vision （WACV）. Piscataway： IEEE Press， 2024： 127-137.
[22]	BERGMANN P， FAUSER M， SATTLEGGER D， et al. MVTec AD： A comprehensive real-world dataset for unsupervised anomaly detection［C］∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. IEEE， 2019： 9592-9600.
[23]	LIU W T， WANG X Y， OWENS J D， et al. Energy-based out-of-distribution detection［C］∥ Proceedings of the 34th International Conference on Neural Information Processing Systems. New York： ACM， 2020： 21464-21475.
[24]	MING Y F， FAN Y， LI Y X. POEM： Out-of-distribution detection with posterior sampling［C］∥ Proceedings of the 39th International Conference on Machine Learning. Baltimore： PMLR， 2022：15650-15665.
[25]	JOSEPH K J， KHAN S， KHAN F S， et al. Towards open world object detection［C］∥ 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 5826-5836.
[26]	WU Z H， LU Y， CHEN X Y， et al. UC-OWOD： Unknown-classified open world object detection［M］∥ Computer Vision-ECCV 2022. Cham： Springer Nature Switzerland， 2022： 193-210.
[27]	GUPTA A， NARAYAN S， JOSEPH K J， et al. OW-DETR： Open-world detection transformer［C］∥ 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 9225-9234.
[28]	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149.
[29]	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 770-778.
[30]	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 936-944.
[31]	WANG Z Y， LI Y， CHEN X， et al. Detecting everything in the open world： Towards universal object detection［C］∥ 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 11433-11443.
[32]	FRAUNDORFER F， ZHANG J， D’URSO M，et al. Semantic drone dataset［DB/OL］. （2019-01-25）［2024-07-01］. .
[33]	ZHANG S F， CHI C， YAO Y Q， et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection［C］∥ 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. IEEE， 2020： 9759-9768.
[34]	JOCHER G， CHAURASIA A， STOKEN A， et al. Ultralytics/YOLOv5： v7.0-YOLOv5 sota realtime instance segmentation［OL］.（2022-11-22）［2024-07-01］. .
[35]	JOCHER G， CHAURASIA A， QIU J. YOLO by Ultralytics （Version 8.0.0）［OL］. （2023-01-10）［2024-07-01］. .
[36]	WANG C Y， YEH I H， MARK LIAO H Y. YOLOv9： Learning what you want to learn using programmable gradient information［M］∥ Computer Vision-ECCV 2024. Cham： Springer Nature Switzerland， 2024： 1-21.
[37]	WANG A， CHEN H， LIU L H， et al. YOLOv10： Real-time end-to-end object detection［OL］. arXiv： 2405， 2024： 14458.

方法	mAP₅₀/%	mAP_50~95/%	Parameters/10⁷	FPS/（frame·s^-1）
Faster R-CNN	92.6	72.5	4.14	17.5
ATSS	93.1	64.7	3.21	20.2
YOLOv5	92.9	60.0	2.17	40.0
YOLOv8	94.2	72.6	2.73	44.8
YOLOv9	95.1	76.5	2.01	29.2
YOLOv10	93.8	75.8	1.54	58.2
YOLARC	92.1	66.2	3.02	37.8
本文方法	95.7	77.5	4.15	17.2

数据集	一致性与否	Recall₅₀/%	AP₅₀/%
测试集	VOS方式（不一致）	30.7	30.0
测试集	本文方式（一致）	98.0	96.1
泛化性测试数据集	VOS方式（不一致）	23.8	22.9
泛化性测试数据集	本文方式（一致）	80.8	48.0

Unknown risk detection in external environment of railroad using UAV images

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 37

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Xiangyu YU, Wen LI, Jie YAN, Shizhe LIANG. Simulation research on thermal management system of fuel cell for liquid hydrogen powered UAV [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 630964-630964.
[2]	Pengqian YANG, Yutong CHEN, Junhui LIU, Jiehao YANG, Jiayuan SHAN, Shijun SUN. Aerodynamic and operational characteristics analysis for tandem wing cargo UAV at high angle of attack [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 131056-131056.
[3]	Rongzu LI, Li LIU, Dun YANG. Optimal design of hydrogen-powered UAV based on multi-source domain fusion surrogate model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 630979-630979.
[4]	Jianyu XU, Li ZHOU, Zhanxue WANG, Jie SHI, Hao SHI. Calculation method for hypersonic plume infrared radiation based on a fast line-by-line calculation model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 630778-630778.
[5]	Kaifang WAN, Zhilin WU, Yunhui WU, Haozhi QIANG, Yibo WU, Bo LI. Cooperative location of multiple UAVs with deep reinforcement learning in GPS-denied environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331024-331024.
[6]	Lingfeng JIANG, Xinkai LI, Hai ZHANG, Hanwei LI, Hongli ZHANG. Mapless navigation of UAVs in dynamic environments based on an improved TD3 algorithm [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331035-331035.
[7]	Lingjie MENG, Hongguang LI, Xinjun LI. SAR image simulation method guided by geomorphic category information [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(7): 331003-331003.
[8]	Jinwu XIANG, Kai MA, Zi KAN, Daochun LI, Kexin ZHENG, Hanxuan CHEN. Review of key technologies for hydrogen powered unmanned aerial vehicles [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(5): 531603-531603.
[9]	Qishuai DING, Bangjun LEI, Zhengping WU. A lightweight single object tracking algorithm for UAVs based on Siamese network [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(4): 330925-330925.
[10]	Zhihao ZHAO, Zhaohua YANG, Yun WU, Yuanjin YU. Single-photon counting imaging denoising method based on deep learning in low-light environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630531-630531.
[11]	Nuo MA, Shechun WEI, Junhui MENG, Qingyang LIU, Yusheng LEI. Flow field characteristics and dynamics of internal supply chamber separating from UAV considering effect of deceleration parachutes [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 130755-130755.
[12]	Yiquan WU, Kang TONG. Research advances on deep learning-based small object detection in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 30848-030848.
[13]	Ruokun QU, Zhiyuan WANG, Yelu LIU, Chenglong LI, Bo JIANG. UAV visual positioning technology for urban air mobility [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531168-531168.
[14]	Weishi CHEN, Hongchuang NIU, Xin WANG, Jian WAN, Xianfeng LU, Jie ZHANG, Qingbin WANG. Review on multi-source detection technologies for birds and drones in airport clearance area [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 31251-031251.
[15]	Jie LIN, Zhigong TANG, Weiqi QIAN, Yueqing WANG, Peng ZHANG, Weixia XU, Jie LIU. Research progress and prospects of aircraft aerodynamic design based on generative models [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 631679-631679.