ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Unknown risk detection in external environment of railroad using UAV images
Received date: 2024-09-24
Revised date: 2024-11-04
Accepted date: 2024-11-25
Online published: 2025-01-16
Supported by
the Fundamental Research Funds for the Central Universities(2024QYBS033);National Key Research and Development Program of China(2022YFB4300600)
Common hazards as well as unknown risks （including mudslides， rockfalls， animal intrusion， etc.） in the external environment of railroad seriously threaten the safe operation of railroads， requiring frequent time-consuming and laborious inspections by inspectors， but the scope of inspections is still very limited. At present， the low-altitude economy has become China’s new quality productivity representative， and the UAV has innate inspection advantages of high altitude， long distance， and small impact from the terrain and railroad maintenance windows. To overcome the challenge of sparse samples and random uncertainty of unknown risks in the external environment of the railroad， this paper utilizes UAVs for remote sensing image acquisition along the railroad， and proposes an unknown risk detection framework based on Faster R-CNN. Firstly， a novel targeted and multi-classification decoupling training strategy is designed and integrated in the unknown risk detection framework， which significantly improves the performance of general object detection and avoids misclassifying unknown risk objects as background. Secondly， the virtual feature synthesis method of VOS（Visual Object Segmentation） is improved， and similarity-based feature space sampling is designed to obtain a generalized unknown risk object feature representation by performing multivariate Gaussian distribution parameter estimation and resampling based on the construction of instance-level object feature space. Subsequently， an energy-based uncertainty measurement is utilized to measure the uncertainty of instance-level features， and losses are calculated accordingly to induce the network to optimize the decision boundaries for common and unknown risk categories. Finally， quantitative and qualitative experimental analyses are conducted on the collected railroad external environment dataset， open-source drone dataset， and generalization test data. The proposed method achieves 95.7% mAP50 in common hazard identification， while achieving 98% and 80.8% Recall50 in the test set and generalization test data， respectively. The experimental results show that the proposed method has high detection ability for unknown risk objects， while ensuring high recognition rate of common hazard categories.
Fanteng MENG , Yong QIN , Jing CUI , Yunpeng WU , Zicheng ZHANG , Shaowei WEI . Unknown risk detection in external environment of railroad using UAV images[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2025 , 46(11) : 531262 -531262 . DOI: 10.7527/S1000-6893.2024.31262
| [1] | 毛天宇. 中国铁路运营里程［OL］. 北京： 新京报， 2024. （2024-01-10） ［2024-07-01］. . |
| MAO T Y. China railway operating mileage［OL］. Beijing： New Beijing News， 2024. （2024-01-10） ［2024-07-01］. （in Chinese）. | |
| [2] | 王薇. 京沪高铁线遭彩钢板撞击［OL］. 北京： 北京青年报， 2018. （2018-08-14） ［2024-07-01］. . |
| WANG W. Beijing-Shanghai High speed rail line hit by color steel plate［OL］. Beijing： Beijing Youth Daily， 2018. （2018-08-14） ［2024-07-01］. （in Chinese）. | |
| [3] | 晓明. 飘物侵袭接触网逼停动车［OL］. 泉州： 泉州网， 2024. （2024-06-27） ［2024-07-01］. . |
| XIAO M. Floating objects invade the catenary and force the train to stop［OL］. Quanzhou： Quanzhou Network， 2024. （2024-06-27） ［2024-07-01］. （in Chinese）. | |
| [4] | 刘楒睿. 泥石流致D2809次列车脱线［OL］. 长沙： 极目新闻， 2022. （2022-06-04） ［2024-07-01］. . |
| LIU S X. Debris flow causing derailment of train D2809［OL］. Changsha： Jimu News， 2022. （2022-06-04） ［2024-07-01］. （in Chinese）. | |
| [5] | 川黔铁路突发山体崩塌落石抢修［OL］. 北京： 中国政府网， 2013. （2013-04-02） ［2024-07-01］. . |
| Sudden mountain collapse and rockfall on the Sichuan Guizhou Railway［OL］. Beijing： Chinese government website， 2013. （2013-04-02） ［2024-07-01］. （in Chinese）. | |
| [6] | WU Y P， CHEN P， QIN Y， et al. Automatic railroad track components inspection using hybrid deep learning framework［J］. IEEE Transactions on Instrumentation and Measurement， 2023， 72： 5011415. |
| [7] | WU Y P， QIN Y， QIAN Y， et al. Hybrid deep learning architecture for rail surface segmentation and surface defect detection［J］. Computer-Aided Civil and Infrastructure Engineering， 2022， 37（2）： 227-244. |
| [8] | CUI J， QIN Y， WU Y P， et al. Skip connection YOLO architecture for noise barrier defect detection using UAV-based images in high-speed railway［J］. IEEE Transactions on Intelligent Transportation Systems， 2023， 24（11）： 12180-12195. |
| [9] | WU Y P， QIN Y， QIAN Y， et al. Automatic detection of arbitrarily oriented fastener defect in high-speed railway［J］. Automation in Construction， 2021， 131： 103913. |
| [10] | WU Y P， MENG F T， QIN Y， et al. UAV imagery based potential safety hazard evaluation for high-speed railroad using Real-time instance segmentation［J］. Advanced Engineering Informatics， 2023， 55： 101819. |
| [11] | TONG L， WANG Z P， JIA L M， et al. Fully decoupled residual ConvNet for real-time railway scene parsing of UAV aerial images［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 23（9）： 14806-14819. |
| [12] | CHEN P， WU Y P， QIN Y， et al. All-in-one YOLO architecture for safety hazard detection of environment along high-speed railway［C］∥ 2022 Global Reliability and Prognostics and Health Management （PHM-Yantai）. Piscataway： IEEE Press， 2022： 1-7. |
| [13] | ZOHAR O， WANG K C， YEUNG S. PROB： Probabilistic objectness for open world object detection［EB/OL］. 2022： 2212.01424. . |
| [14] | DU X F， WANG Z N， CAI M， et al. VOS： Learning what you don’t know by virtual outlier synthesis［EB/OL］. 2022： 2202.01197. . |
| [15] | WU A M， DENG C. TIB： Detecting unknown objects via two-stream information bottleneck［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2024， 46（1）： 611-625. |
| [16] | TONG L， JIA L M， GENG Y X， et al. Anchor-adaptive railway track detection from unmanned aerial vehicle images［J］. Computer-Aided Civil and Infrastructure Engineering， 2023， 38（18）： 2666-2684. |
| [17] | MU Z H， QIN Y， YU C C， et al. Adaptive cropping shallow attention network for defect detection of bridge girder steel using unmanned aerial vehicle images［J］. Journal of Zhejiang University： Science A， 2023， 24（3）： 243-256. |
| [18] | GUDOVSKIY D， ISHIZAKA S， KOZUKA K. CFLOW-AD： Real-time unsupervised anomaly detection with localization via conditional normalizing flows［C］∥ 2022 IEEE/CVF Winter Conference on Applications of Computer Vision （WACV）. Piscataway： IEEE Press， 2022： 1819-1828. |
| [19] | YU J W， ZHENG Y， WANG X， et al. FastFlow： Unsupervised anomaly detection and localization via 2D normalizing flows［EB/OL］. 2021： 2111.07677. . |
| [20] | YOU Z Y， CUI L， SHEN Y J， et al. A unified model for multi-class anomaly detection［C］∥ 36th Conference on Neural Information Processing Systems （NeurIPS 2022）. New Orleans： NeurIPS Proceedings， 2022： 1-14. |
| [21] | BATZNER K， HECKLER L， K?NIG R. EfficientAD： Accurate visual anomaly detection at millisecond-level latencies［C］∥ 2024 IEEE/CVF Winter Conference on Applications of Computer Vision （WACV）. Piscataway： IEEE Press， 2024： 127-137. |
| [22] | BERGMANN P， FAUSER M， SATTLEGGER D， et al. MVTec AD： A comprehensive real-world dataset for unsupervised anomaly detection［C］∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. IEEE， 2019： 9592-9600. |
| [23] | LIU W T， WANG X Y， OWENS J D， et al. Energy-based out-of-distribution detection［C］∥ Proceedings of the 34th International Conference on Neural Information Processing Systems. New York： ACM， 2020： 21464-21475. |
| [24] | MING Y F， FAN Y， LI Y X. POEM： Out-of-distribution detection with posterior sampling［C］∥ Proceedings of the 39th International Conference on Machine Learning. Baltimore： PMLR， 2022：15650-15665. |
| [25] | JOSEPH K J， KHAN S， KHAN F S， et al. Towards open world object detection［C］∥ 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 5826-5836. |
| [26] | WU Z H， LU Y， CHEN X Y， et al. UC-OWOD： Unknown-classified open world object detection［M］∥ Computer Vision-ECCV 2022. Cham： Springer Nature Switzerland， 2022： 193-210. |
| [27] | GUPTA A， NARAYAN S， JOSEPH K J， et al. OW-DETR： Open-world detection transformer［C］∥ 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 9225-9234. |
| [28] | REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149. |
| [29] | HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 770-778. |
| [30] | LIN T Y， DOLLáR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 936-944. |
| [31] | WANG Z Y， LI Y， CHEN X， et al. Detecting everything in the open world： Towards universal object detection［C］∥ 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 11433-11443. |
| [32] | FRAUNDORFER F， ZHANG J， D’URSO M，et al. Semantic drone dataset［DB/OL］. （2019-01-25）［2024-07-01］. . |
| [33] | ZHANG S F， CHI C， YAO Y Q， et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection［C］∥ 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. IEEE， 2020： 9759-9768. |
| [34] | JOCHER G， CHAURASIA A， STOKEN A， et al. Ultralytics/YOLOv5： v7.0-YOLOv5 sota realtime instance segmentation［OL］.（2022-11-22）［2024-07-01］. . |
| [35] | JOCHER G， CHAURASIA A， QIU J. YOLO by Ultralytics （Version 8.0.0）［OL］. （2023-01-10）［2024-07-01］. . |
| [36] | WANG C Y， YEH I H， MARK LIAO H Y. YOLOv9： Learning what you want to learn using programmable gradient information［M］∥ Computer Vision-ECCV 2024. Cham： Springer Nature Switzerland， 2024： 1-21. |
| [37] | WANG A， CHEN H， LIU L H， et al. YOLOv10： Real-time end-to-end object detection［OL］. arXiv： 2405， 2024： 14458. |
/
| 〈 |
|
〉 |
CJA