论文

铁路外部环境无人机图像未知风险检测方法

  • 孟凡腾 ,
  • 秦勇 ,
  • 崔京 ,
  • 吴云鹏 ,
  • 张紫城 ,
  • 魏少伟
展开
  • 1.北京交通大学 先进轨道交通自主运行全国重点实验室,北京 100044
    2.北京交通大学 运营主动安全保障与风险防控铁路行业重点实验室,北京 100044
    3.昆明理工大学 交通工程学院,昆明 650031
    4.中国铁道科学研究院集团有限公司 铁道建筑研究所,北京 100081
.E-mail: yqin@bjtu.edu.cn

收稿日期: 2024-09-24

  修回日期: 2024-11-04

  录用日期: 2024-11-25

  网络出版日期: 2025-01-16

基金资助

中央高校基本科研业务费专项(2024QYBS033);国家重点研发计划(2022YFB4300600)

Unknown risk detection in external environment of railroad using UAV images

  • Fanteng MENG ,
  • Yong QIN ,
  • Jing CUI ,
  • Yunpeng WU ,
  • Zicheng ZHANG ,
  • Shaowei WEI
Expand
  • 1.State Key Laboratory of Advanced Rail Autonomous Operation,Beijing Jiaotong University,Beijing 100044,China
    2.Key Laboratory of Railway Industry of Proactive Safety and Risk Control,Beijing Jiaotong University,Beijing 100044,China
    3.Faculty of Transportation Engineering,Kunming University of Science and Technology,Kunming 650031,China
    4.Railway Engineering Research Institute,China Academy of Railway Sciences Corporation Limited,Beijing 100081,China
E-mail: yqin@bjtu.edu.cn

Received date: 2024-09-24

  Revised date: 2024-11-04

  Accepted date: 2024-11-25

  Online published: 2025-01-16

Supported by

the Fundamental Research Funds for the Central Universities(2024QYBS033);National Key Research and Development Program of China(2022YFB4300600)

摘要

铁路外部环境的常见隐患以及未知风险(包括泥石流、落石、动物入侵等)严重威胁着铁路安全运行,需要巡检人员耗时费力地频繁检查,但巡检范围仍十分有限。目前,低空经济已成为国家新质生产力代表,无人机凭借其高空作业能力、远距离覆盖优势,以及不受地形限制和铁路维修天窗影响的特点,在巡检领域具有得天独厚的技术优势。针对铁路外部环境未知风险样本稀疏且具有随机不确定性的挑战,利用无人机进行沿线遥感图像采集,并基于Faster R-CNN提出了一种未知风险检测框架。首先,设计了一种目标性与多分类解耦训练策略,并融合在未知风险检测框架中,显著提升了通用目标检测性能,避免将未知风险目标错分为背景。其次,改进了VOS的虚拟特征合成方法,提出了基于相似度的特征空间采样,在构建实例级目标特征空间基础上进行多元高斯分布参数估计与重采样,获得泛化性的未知风险目标特征表示。再次,利用基于能量的不确定性度量,对实例级特征进行不确定性度量,并据此计算损失以诱导网络优化常见类别和未知风险类别的决策边界。最后,在采集的铁路外部环境数据集、开源无人机数据集以及泛化性测试数据上进行了定量与定性实验分析,本文方法在常见隐患识别上取得了95.7%的mAP50,同时在测试集和泛化性测试数据上分别取得了98%和80.8%的Recall50,实验结果表明本文方法在保证常见隐患类别高识别率的基础上对于未知风险目标也有较高的检测能力。

本文引用格式

孟凡腾 , 秦勇 , 崔京 , 吴云鹏 , 张紫城 , 魏少伟 . 铁路外部环境无人机图像未知风险检测方法[J]. 航空学报, 2025 , 46(11) : 531262 -531262 . DOI: 10.7527/S1000-6893.2024.31262

Abstract

Common hazards as well as unknown risks (including mudslides, rockfalls, animal intrusion, etc.) in the external environment of railroad seriously threaten the safe operation of railroads, requiring frequent time-consuming and laborious inspections by inspectors, but the scope of inspections is still very limited. At present, the low-altitude economy has become China’s new quality productivity representative, and the UAV has innate inspection advantages of high altitude, long distance, and small impact from the terrain and railroad maintenance windows. To overcome the challenge of sparse samples and random uncertainty of unknown risks in the external environment of the railroad, this paper utilizes UAVs for remote sensing image acquisition along the railroad, and proposes an unknown risk detection framework based on Faster R-CNN. Firstly, a novel targeted and multi-classification decoupling training strategy is designed and integrated in the unknown risk detection framework, which significantly improves the performance of general object detection and avoids misclassifying unknown risk objects as background. Secondly, the virtual feature synthesis method of VOS(Visual Object Segmentation) is improved, and similarity-based feature space sampling is designed to obtain a generalized unknown risk object feature representation by performing multivariate Gaussian distribution parameter estimation and resampling based on the construction of instance-level object feature space. Subsequently, an energy-based uncertainty measurement is utilized to measure the uncertainty of instance-level features, and losses are calculated accordingly to induce the network to optimize the decision boundaries for common and unknown risk categories. Finally, quantitative and qualitative experimental analyses are conducted on the collected railroad external environment dataset, open-source drone dataset, and generalization test data. The proposed method achieves 95.7% mAP50 in common hazard identification, while achieving 98% and 80.8% Recall50 in the test set and generalization test data, respectively. The experimental results show that the proposed method has high detection ability for unknown risk objects, while ensuring high recognition rate of common hazard categories.

参考文献

[1] 毛天宇. 中国铁路运营里程[OL]. 北京: 新京报2024. (2024-01-10) [2024-07-01]. .
  MAO T Y. China railway operating mileage[OL]. Beijing: New Beijing News, 2024. (2024-01-10) [2024-07-01]. (in Chinese).
[2] 王薇. 京沪高铁线遭彩钢板撞击[OL]. 北京: 北京青年报2018. (2018-08-14) [2024-07-01]. .
  WANG W. Beijing-Shanghai High speed rail line hit by color steel plate[OL]. Beijing: Beijing Youth Daily, 2018. (2018-08-14) [2024-07-01]. (in Chinese).
[3] 晓明. 飘物侵袭接触网逼停动车[OL]. 泉州: 泉州网2024. (2024-06-27) [2024-07-01]. .
  XIAO M. Floating objects invade the catenary and force the train to stop[OL]. Quanzhou: Quanzhou Network2024. (2024-06-27) [2024-07-01]. (in Chinese).
[4] 刘楒睿. 泥石流致D2809次列车脱线[OL]. 长沙: 极目新闻2022. (2022-06-04) [2024-07-01]. .
  LIU S X. Debris flow causing derailment of train D2809[OL]. Changsha: Jimu News2022. (2022-06-04) [2024-07-01]. (in Chinese).
[5] 川黔铁路突发山体崩塌落石抢修[OL]. 北京: 中国政府网2013. (2013-04-02) [2024-07-01]. .
  Sudden mountain collapse and rockfall on the Sichuan Guizhou Railway[OL]. Beijing: Chinese government website, 2013. (2013-04-02) [2024-07-01]. (in Chinese).
[6] WU Y P, CHEN P, QIN Y, et al. Automatic railroad track components inspection using hybrid deep learning framework[J]. IEEE Transactions on Instrumentation and Measurement202372: 5011415.
[7] WU Y P, QIN Y, QIAN Y, et al. Hybrid deep learning architecture for rail surface segmentation and surface defect detection[J]. Computer-Aided Civil and Infrastructure Engineering202237(2): 227-244.
[8] CUI J, QIN Y, WU Y P, et al. Skip connection YOLO architecture for noise barrier defect detection using UAV-based images in high-speed railway[J]. IEEE Transactions on Intelligent Transportation Systems202324(11): 12180-12195.
[9] WU Y P, QIN Y, QIAN Y, et al. Automatic detection of arbitrarily oriented fastener defect in high-speed railway[J]. Automation in Construction2021131: 103913.
[10] WU Y P, MENG F T, QIN Y, et al. UAV imagery based potential safety hazard evaluation for high-speed railroad using Real-time instance segmentation[J]. Advanced Engineering Informatics202355: 101819.
[11] TONG L, WANG Z P, JIA L M, et al. Fully decoupled residual ConvNet for real-time railway scene parsing of UAV aerial images[J]. IEEE Transactions on Intelligent Transportation Systems202223(9): 14806-14819.
[12] CHEN P, WU Y P, QIN Y, et al. All-in-one YOLO architecture for safety hazard detection of environment along high-speed railway[C]∥ 2022 Global Reliability and Prognostics and Health Management (PHM-Yantai). Piscataway: IEEE Press, 2022: 1-7.
[13] ZOHAR O, WANG K C, YEUNG S. PROB: Probabilistic objectness for open world object detection[EB/OL]. 2022: 2212.01424. .
[14] DU X F, WANG Z N, CAI M, et al. VOS: Learning what you don’t know by virtual outlier synthesis[EB/OL]. 2022: 2202.01197. .
[15] WU A M, DENG C. TIB: Detecting unknown objects via two-stream information bottleneck[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence202446(1): 611-625.
[16] TONG L, JIA L M, GENG Y X, et al. Anchor-adaptive railway track detection from unmanned aerial vehicle images[J]. Computer-Aided Civil and Infrastructure Engineering202338(18): 2666-2684.
[17] MU Z H, QIN Y, YU C C, et al. Adaptive cropping shallow attention network for defect detection of bridge girder steel using unmanned aerial vehicle images[J]. Journal of Zhejiang University: Science A202324(3): 243-256.
[18] GUDOVSKIY D, ISHIZAKA S, KOZUKA K. CFLOW-AD: Real-time unsupervised anomaly detection with localization via conditional normalizing flows[C]∥ 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2022: 1819-1828.
[19] YU J W, ZHENG Y, WANG X, et al. FastFlow: Unsupervised anomaly detection and localization via 2D normalizing flows[EB/OL]. 2021: 2111.07677. .
[20] YOU Z Y, CUI L, SHEN Y J, et al. A unified model for multi-class anomaly detection[C]∥ 36th Conference on Neural Information Processing Systems (NeurIPS 2022). New Orleans: NeurIPS Proceedings, 2022: 1-14.
[21] BATZNER K, HECKLER L, K?NIG R. EfficientAD: Accurate visual anomaly detection at millisecond-level latencies[C]∥ 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2024: 127-137.
[22] BERGMANN P, FAUSER M, SATTLEGGER D, et al. MVTec AD: A comprehensive real-world dataset for unsupervised anomaly detection[C]∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019: 9592-9600.
[23] LIU W T, WANG X Y, OWENS J D, et al. Energy-based out-of-distribution detection[C]∥ Proceedings of the 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 21464-21475.
[24] MING Y F, FAN Y, LI Y X. POEM: Out-of-distribution detection with posterior sampling[C]∥ Proceedings of the 39th International Conference on Machine Learning. Baltimore: PMLR, 2022:15650-15665.
[25] JOSEPH K J, KHAN S, KHAN F S, et al. Towards open world object detection[C]∥ 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2021: 5826-5836.
[26] WU Z H, LU Y, CHEN X Y, et al. UC-OWOD: Unknown-classified open world object detection[M]∥ Computer Vision-ECCV 2022. Cham: Springer Nature Switzerland, 2022: 193-210.
[27] GUPTA A, NARAYAN S, JOSEPH K J, et al. OW-DETR: Open-world detection transformer[C]∥ 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2022: 9225-9234.
[28] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence201739(6): 1137-1149.
[29] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2016: 770-778.
[30] LIN T Y, DOLLáR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2017: 936-944.
[31] WANG Z Y, LI Y, CHEN X, et al. Detecting everything in the open world: Towards universal object detection[C]∥ 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2023: 11433-11443.
[32] FRAUNDORFER F, ZHANG J, D’URSO M,et al. Semantic drone dataset[DB/OL]. (2019-01-25)[2024-07-01]. .
[33] ZHANG S F, CHI C, YAO Y Q, et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection[C]∥ 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 9759-9768.
[34] JOCHER G, CHAURASIA A, STOKEN A, et al. Ultralytics/YOLOv5: v7.0-YOLOv5 sota realtime instance segmentation[OL].(2022-11-22)[2024-07-01]. .
[35] JOCHER G, CHAURASIA A, QIU J. YOLO by Ultralytics (Version 8.0.0)[OL]. (2023-01-10)[2024-07-01]. .
[36] WANG C Y, YEH I H, MARK LIAO H Y. YOLOv9: Learning what you want to learn using programmable gradient information[M]∥ Computer Vision-ECCV 2024. Cham: Springer Nature Switzerland, 2024: 1-21.
[37] WANG A, CHEN H, LIU L H, et al. YOLOv10: Real-time end-to-end object detection[OL]. arXiv: 24052024: 14458.
文章导航

/