铁路外部环境无人机图像未知风险检测方法
收稿日期: 2024-09-24
修回日期: 2024-11-04
录用日期: 2024-11-25
网络出版日期: 2025-01-16
基金资助
中央高校基本科研业务费专项(2024QYBS033);国家重点研发计划(2022YFB4300600)
Unknown risk detection in external environment of railroad using UAV images
Received date: 2024-09-24
Revised date: 2024-11-04
Accepted date: 2024-11-25
Online published: 2025-01-16
Supported by
the Fundamental Research Funds for the Central Universities(2024QYBS033);National Key Research and Development Program of China(2022YFB4300600)
铁路外部环境的常见隐患以及未知风险(包括泥石流、落石、动物入侵等)严重威胁着铁路安全运行,需要巡检人员耗时费力地频繁检查,但巡检范围仍十分有限。目前,低空经济已成为国家新质生产力代表,无人机凭借其高空作业能力、远距离覆盖优势,以及不受地形限制和铁路维修天窗影响的特点,在巡检领域具有得天独厚的技术优势。针对铁路外部环境未知风险样本稀疏且具有随机不确定性的挑战,利用无人机进行沿线遥感图像采集,并基于Faster R-CNN提出了一种未知风险检测框架。首先,设计了一种目标性与多分类解耦训练策略,并融合在未知风险检测框架中,显著提升了通用目标检测性能,避免将未知风险目标错分为背景。其次,改进了VOS的虚拟特征合成方法,提出了基于相似度的特征空间采样,在构建实例级目标特征空间基础上进行多元高斯分布参数估计与重采样,获得泛化性的未知风险目标特征表示。再次,利用基于能量的不确定性度量,对实例级特征进行不确定性度量,并据此计算损失以诱导网络优化常见类别和未知风险类别的决策边界。最后,在采集的铁路外部环境数据集、开源无人机数据集以及泛化性测试数据上进行了定量与定性实验分析,本文方法在常见隐患识别上取得了95.7%的mAP50,同时在测试集和泛化性测试数据上分别取得了98%和80.8%的Recall50,实验结果表明本文方法在保证常见隐患类别高识别率的基础上对于未知风险目标也有较高的检测能力。
孟凡腾 , 秦勇 , 崔京 , 吴云鹏 , 张紫城 , 魏少伟 . 铁路外部环境无人机图像未知风险检测方法[J]. 航空学报, 2025 , 46(11) : 531262 -531262 . DOI: 10.7527/S1000-6893.2024.31262
Common hazards as well as unknown risks (including mudslides, rockfalls, animal intrusion, etc.) in the external environment of railroad seriously threaten the safe operation of railroads, requiring frequent time-consuming and laborious inspections by inspectors, but the scope of inspections is still very limited. At present, the low-altitude economy has become China’s new quality productivity representative, and the UAV has innate inspection advantages of high altitude, long distance, and small impact from the terrain and railroad maintenance windows. To overcome the challenge of sparse samples and random uncertainty of unknown risks in the external environment of the railroad, this paper utilizes UAVs for remote sensing image acquisition along the railroad, and proposes an unknown risk detection framework based on Faster R-CNN. Firstly, a novel targeted and multi-classification decoupling training strategy is designed and integrated in the unknown risk detection framework, which significantly improves the performance of general object detection and avoids misclassifying unknown risk objects as background. Secondly, the virtual feature synthesis method of VOS(Visual Object Segmentation) is improved, and similarity-based feature space sampling is designed to obtain a generalized unknown risk object feature representation by performing multivariate Gaussian distribution parameter estimation and resampling based on the construction of instance-level object feature space. Subsequently, an energy-based uncertainty measurement is utilized to measure the uncertainty of instance-level features, and losses are calculated accordingly to induce the network to optimize the decision boundaries for common and unknown risk categories. Finally, quantitative and qualitative experimental analyses are conducted on the collected railroad external environment dataset, open-source drone dataset, and generalization test data. The proposed method achieves 95.7% mAP50 in common hazard identification, while achieving 98% and 80.8% Recall50 in the test set and generalization test data, respectively. The experimental results show that the proposed method has high detection ability for unknown risk objects, while ensuring high recognition rate of common hazard categories.
[1] | 毛天宇. 中国铁路运营里程[OL]. 北京: 新京报, 2024. (2024-01-10) [2024-07-01]. . |
MAO T Y. China railway operating mileage[OL]. Beijing: New Beijing News, 2024. (2024-01-10) [2024-07-01]. (in Chinese). | |
[2] | 王薇. 京沪高铁线遭彩钢板撞击[OL]. 北京: 北京青年报, 2018. (2018-08-14) [2024-07-01]. . |
WANG W. Beijing-Shanghai High speed rail line hit by color steel plate[OL]. Beijing: Beijing Youth Daily, 2018. (2018-08-14) [2024-07-01]. (in Chinese). | |
[3] | 晓明. 飘物侵袭接触网逼停动车[OL]. 泉州: 泉州网, 2024. (2024-06-27) [2024-07-01]. . |
XIAO M. Floating objects invade the catenary and force the train to stop[OL]. Quanzhou: Quanzhou Network, 2024. (2024-06-27) [2024-07-01]. (in Chinese). | |
[4] | 刘楒睿. 泥石流致D2809次列车脱线[OL]. 长沙: 极目新闻, 2022. (2022-06-04) [2024-07-01]. . |
LIU S X. Debris flow causing derailment of train D2809[OL]. Changsha: Jimu News, 2022. (2022-06-04) [2024-07-01]. (in Chinese). | |
[5] | 川黔铁路突发山体崩塌落石抢修[OL]. 北京: 中国政府网, 2013. (2013-04-02) [2024-07-01]. . |
Sudden mountain collapse and rockfall on the Sichuan Guizhou Railway[OL]. Beijing: Chinese government website, 2013. (2013-04-02) [2024-07-01]. (in Chinese). | |
[6] | WU Y P, CHEN P, QIN Y, et al. Automatic railroad track components inspection using hybrid deep learning framework[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 5011415. |
[7] | WU Y P, QIN Y, QIAN Y, et al. Hybrid deep learning architecture for rail surface segmentation and surface defect detection[J]. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(2): 227-244. |
[8] | CUI J, QIN Y, WU Y P, et al. Skip connection YOLO architecture for noise barrier defect detection using UAV-based images in high-speed railway[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(11): 12180-12195. |
[9] | WU Y P, QIN Y, QIAN Y, et al. Automatic detection of arbitrarily oriented fastener defect in high-speed railway[J]. Automation in Construction, 2021, 131: 103913. |
[10] | WU Y P, MENG F T, QIN Y, et al. UAV imagery based potential safety hazard evaluation for high-speed railroad using Real-time instance segmentation[J]. Advanced Engineering Informatics, 2023, 55: 101819. |
[11] | TONG L, WANG Z P, JIA L M, et al. Fully decoupled residual ConvNet for real-time railway scene parsing of UAV aerial images[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(9): 14806-14819. |
[12] | CHEN P, WU Y P, QIN Y, et al. All-in-one YOLO architecture for safety hazard detection of environment along high-speed railway[C]∥ 2022 Global Reliability and Prognostics and Health Management (PHM-Yantai). Piscataway: IEEE Press, 2022: 1-7. |
[13] | ZOHAR O, WANG K C, YEUNG S. PROB: Probabilistic objectness for open world object detection[EB/OL]. 2022: 2212.01424. . |
[14] | DU X F, WANG Z N, CAI M, et al. VOS: Learning what you don’t know by virtual outlier synthesis[EB/OL]. 2022: 2202.01197. . |
[15] | WU A M, DENG C. TIB: Detecting unknown objects via two-stream information bottleneck[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(1): 611-625. |
[16] | TONG L, JIA L M, GENG Y X, et al. Anchor-adaptive railway track detection from unmanned aerial vehicle images[J]. Computer-Aided Civil and Infrastructure Engineering, 2023, 38(18): 2666-2684. |
[17] | MU Z H, QIN Y, YU C C, et al. Adaptive cropping shallow attention network for defect detection of bridge girder steel using unmanned aerial vehicle images[J]. Journal of Zhejiang University: Science A, 2023, 24(3): 243-256. |
[18] | GUDOVSKIY D, ISHIZAKA S, KOZUKA K. CFLOW-AD: Real-time unsupervised anomaly detection with localization via conditional normalizing flows[C]∥ 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2022: 1819-1828. |
[19] | YU J W, ZHENG Y, WANG X, et al. FastFlow: Unsupervised anomaly detection and localization via 2D normalizing flows[EB/OL]. 2021: 2111.07677. . |
[20] | YOU Z Y, CUI L, SHEN Y J, et al. A unified model for multi-class anomaly detection[C]∥ 36th Conference on Neural Information Processing Systems (NeurIPS 2022). New Orleans: NeurIPS Proceedings, 2022: 1-14. |
[21] | BATZNER K, HECKLER L, K?NIG R. EfficientAD: Accurate visual anomaly detection at millisecond-level latencies[C]∥ 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2024: 127-137. |
[22] | BERGMANN P, FAUSER M, SATTLEGGER D, et al. MVTec AD: A comprehensive real-world dataset for unsupervised anomaly detection[C]∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019: 9592-9600. |
[23] | LIU W T, WANG X Y, OWENS J D, et al. Energy-based out-of-distribution detection[C]∥ Proceedings of the 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 21464-21475. |
[24] | MING Y F, FAN Y, LI Y X. POEM: Out-of-distribution detection with posterior sampling[C]∥ Proceedings of the 39th International Conference on Machine Learning. Baltimore: PMLR, 2022:15650-15665. |
[25] | JOSEPH K J, KHAN S, KHAN F S, et al. Towards open world object detection[C]∥ 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2021: 5826-5836. |
[26] | WU Z H, LU Y, CHEN X Y, et al. UC-OWOD: Unknown-classified open world object detection[M]∥ Computer Vision-ECCV 2022. Cham: Springer Nature Switzerland, 2022: 193-210. |
[27] | GUPTA A, NARAYAN S, JOSEPH K J, et al. OW-DETR: Open-world detection transformer[C]∥ 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2022: 9225-9234. |
[28] | REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. |
[29] | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2016: 770-778. |
[30] | LIN T Y, DOLLáR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2017: 936-944. |
[31] | WANG Z Y, LI Y, CHEN X, et al. Detecting everything in the open world: Towards universal object detection[C]∥ 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2023: 11433-11443. |
[32] | FRAUNDORFER F, ZHANG J, D’URSO M,et al. Semantic drone dataset[DB/OL]. (2019-01-25)[2024-07-01]. . |
[33] | ZHANG S F, CHI C, YAO Y Q, et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection[C]∥ 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 9759-9768. |
[34] | JOCHER G, CHAURASIA A, STOKEN A, et al. Ultralytics/YOLOv5: v7.0-YOLOv5 sota realtime instance segmentation[OL].(2022-11-22)[2024-07-01]. . |
[35] | JOCHER G, CHAURASIA A, QIU J. YOLO by Ultralytics (Version 8.0.0)[OL]. (2023-01-10)[2024-07-01]. . |
[36] | WANG C Y, YEH I H, MARK LIAO H Y. YOLOv9: Learning what you want to learn using programmable gradient information[M]∥ Computer Vision-ECCV 2024. Cham: Springer Nature Switzerland, 2024: 1-21. |
[37] | WANG A, CHEN H, LIU L H, et al. YOLOv10: Real-time end-to-end object detection[OL]. arXiv: 2405, 2024: 14458. |
/
〈 |
|
〉 |