基于贝叶斯优化的机载智能避让系统安全性评估

doi:10.7527/S1000-6893.2025.31973

Abstract

Abstract:

To address the airworthiness safety challenges brought by the application of reinforcement learning in UAV intelligent avoidance systems， this paper proposes a safety assessment method for the intelligent avoidance system based on Bayesian optimization theory within the framework of the SAE ARP4761 standard. First， the intelligent avoidance system model is established based on the UAV kinematic model and the Proximal Policy Optimization （PPO） algorithm. Second， by integrating the system model verification task with Bayesian optimization theory， the iterative training of the Gaussian surrogate model is achieved through three acquisition functions： uncertainty exploration， boundary refinement， and failure region sampling. This enables safety verification， safety boundary determination， and functional failure probability analysis of the intelligent avoidance system with a small number of samples， supporting quantitative safety assessment at the whole aircraft/system level. Finally， taking a typical intelligent avoidance system architecture as a case， the proposed method is demonstrated to effectively support airworthiness safety assessment， providing essential airworthiness compliance methods and technical guarantees for the deployment of intelligent avoidance systems. Experimental results further validate that， under limited sample conditions， the Bayesian optimization-based method outperforms uniform sampling and Monte Carlo methods by offering more detailed failure boundary predictions， precise failure probability estimation， and higher confidence levels for the reinforcement learning module.

Key words: reinforcement learning, airborne intelligent avoidance system, proximal policy optimization, Bayesian optimization, airworthiness safety

CLC Number:

V244.12

Zan MA, Jie BAI, Liqin YAN, Yong CHEN, Shuguang SUN. Safety assessment for airborne intelligent avoidance system based on Bayesian optimization[J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(1): 331973.

Figures/Tables 21

Fig.1

Fig.2

Table 1

Safety assessment process of intelligent collision avoidance system with RL models

步骤

活动

步骤1：功能危害评估

在运行概念下进行系统功能危害分析（本文重点关注含RL模型错误“机动”功能危害）

步骤2：初步系统安全性评估

1.定义安全性目标

2.定义初步系统架构以满足安全性目标

3.衍生包括独立需求的安全性需求，满足目标和支持架构

4.定义和确认假设

5.分配研制保证水平（DAL）

6.基于贝叶斯优化，通过不确定性探索、边界细化和失效区域采样函数，训练高斯代理模型

7.在冲突距离和冲突角度两维输入空间X和分布函数 $p x$ 的场景下对RL模型进行失效概率估计与分析

8.衍生需求满足性的RL模型运行域（即安全边界）

9.执行RL单元失效模式影响分析

步骤3：系统安全性评估

执行最终的安全性评估

Table 1

Fig. 3

Fig. 4

Fig.5

Fig.6

Fig.7

Table 2

Four evaluation methods for safety verification tasks

评估方法	$R f a i l$	$x *$	$p x *$	$P ̂ f a i l$	$Δ P f a i l$
真实系统	20.8%	（0.867 93，1.895 79）	1.496×10^-3	8.410 72×10^-5
代理模型	50.2%	（0.861 72，1.895 79）	1.396×10^-3	8.407 26×10^-5	4.12×10^-4
均匀采样	21.7%	（0.838 70，1.935 48）	9.543×10^-4	8.055 44×10^-5	4.22×10^-2
蒙特卡洛	21.6%	（0.827 56，1.918 87）	9.687×10^-4	8.619 19×10^-5	2.48×10^-2

Table 2

Fig.8

Fig.9

Fig.10

Table 3

Ablation study of three acquisition functions for safety validation task

获取函数组合	$R f a i l$	$x *$	$p x *$	$P ̂ f a i l$	$Δ P f a i l$
不确定性探索	21.6%	（0.833 67，1.907 82）	1.030×10^-3	8.522 94×10^-5	1.33×10^-2
边界细化	35.5%	（0.861 72，1.899 80）	1.368×10^-3	8.628 98×10^-5	2.59×10^-2
失效区域采样	89.2%	（0.436 88，1.330 66）	3.352×10^-4	5.827 44×10^-5	3.07×10^-1
不确定性探索、边界细化	34.6%	（0.857 72，1.895 79）	1.366×10^-3	8.485 22×10^-5	8.86×10^-3
不确定性探索、失效区域采样	49.3%	（0.821 64，1.887 77）	1.094×10^-3	7.749 55×10^-5	7.86×10^-2
边界细化、失效区域采样	52.8%	（0.857 71，1.895 79）	1.361×10^-3	8.453 47×10^-5	5.08×10^-3
代理模型	50.2%	（0.861 72，1.895 79）	1.396×10^-3	8.407 26×10^-5	4.12×10^-4

Table 3

Fig.11

Table 4

Comparison of safety verification tasks in extended experiments

评估方法	$R f a i l$	$x *$	$p x *$	$P ̂ f a i l$
均匀采样	21.4%	（0.915，1.932，1.254）	1.665×10^-3	8.051 64×10^-5
代理模型	47.2%	（0.929，1.939，1.293）	1.331×10^-3	7.714 31×10^-5

Table 4

Fig.12

Fig.13

Fig.14

Fig.15

Table 5

Fig.16

References 32

[1]	DIEZ-TOMILLO J， ALCARAZ-CALERO J M， WANG Q. Face verification algorithms for UAV applications： An empirical comparative analysis［J］. Journal of Communications Software and Systems， 2024， 20（1）： 1-12.
[2]	张学军，李诚龙，张志远，等. 低空航行系统实时风险管理能力构建：概念、挑战与技术［J］. 航空学报， 2025， 46（11）： 8-34.
	ZHANG X J， LI C L， ZHANG Z Y， et al. Constructing in-time risk management capabilities for low-altitude aviation systems： Concepts， technologies， and challenges［J］. Acta Aeronautica et Astronautica Sinica， 2025， 46（11）： 8-34 （in Chinese）.
[3]	国家市场监督管理总局国家标准化管理委员会. 民用无人驾驶航空器系统安全要求：［S］. 北京：中国标准出版社， 2023.
	Standardization Administration of the People’s Republic of China. Safety requirements for civil unmanned aircraft system：［S］. Beijing： Standards Press of China， 2023 （in Chinese）.
[4]	HARMAN W H. TCAS—A system for preventing midair collisions［J］. The Lincoln Laboratory Journal， 1989， 2（3）： 437–457.
[5]	KOCHENDERFER M J， HOLLAND J E， CHRYSSANTHACOPOULOS J P. Next-generation airborne collision avoidance system［J］. Lincoln Laboratory Journal， 2012， 19（1）： 17-33.
[6]	KOCHENDERFER M J， AMATO C， CHOWDHARY G， et al. Optimized airborne collision avoidance［M］∥Decision Making under Uncertainty： Theory and Application. Cambridge： MIT Press， 2015： 249-276.
[7]	ARULKUMARAN K， DEISENROTH M P， BRUNDAGE M， et al. Deep reinforcement learning： A brief survey［J］. IEEE Signal Processing Magazine， 2017， 34（6）： 26-38.
[8]	WULFE B. UAV collision avoidance policy optimization with deep reinforcement learning［D］. Palo Alto：Stanford University， 2017.
[9]	汤新民，李帅，顾俊伟，等. 一种无人机冲突探测与避让系统决策方法［J］. 电子与信息学报， 2025， 47（5）： 1301-1309.
	TANG X M， LI S， GU J W， et al. A decision-making method for UAV conflict detection and avoidance system［J］. Journal of Electronics & Information Technology， 2025， 47（5）： 1301-1309 （in Chinese）.
[10]	HU J M， YANG X X， WANG W C， et al. Obstacle avoidance for UAS in continuous action space using deep reinforcement learning［J］. IEEE Access， 2022， 10： 90623-90634.
[11]	马赞，白杰，陈勇，等. 基于条件高斯PAC-Bayes的机载CNN分类器安全性评估［J］. 航空学报， 2025， 46（4）： 330824.
	MA Z， BAI J， CHEN Y， et al. Safety assessment for airborne CNN classifier based on conditional Gaussian PAC-Bayes［J］. Acta Aeronautica et Astronautica Sinica， 2025， 46（4）： 330824 （in Chinese）.
[12]	International Recommended Practice SAE. Guidelines and methods for conducting the safety assessment process on civil airborne systems and equipment： ARP4761 ［S］. Warrendale： SAE International， 1996.
[13]	CLAVIÈRE A， ASSELIN E， GARION C， et al. Safety verification of neural network controlled systems［C］∥2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops （DSN-W）. Piscataway： IEEE Press， 2021： 47-54.
[14]	BAK S. Nnenum： Verification of ReLU neural networks with optimized abstraction refinement［C］∥NASA Formal Methods. Cham： Springer， 2021： 19-36.
[15]	LECUN Y， CORTES C， BURGES C J C. The MNIST database of handwritten digits［EB/OL］. （2013-05-14）［2025-03-01］. .
[16]	KRIZHEVSKY A， HINTON G. Learning multiple layers of features from tiny images［J］. Handbook of Systemic Autoimmune Diseases， 2009， 1（4）： 1-57.
[17]	MAI V， MANI K， PAULL L. Sample efficient deep reinforcement learning via uncertainty estimation［DB/OL］. arXiv preprint： 2201.01666， 2022.
[18]	PUSTYNNIKOV A， EREMEEV D. Estimating uncertainty for vehicle motion prediction on yandex shifts dataset［DB/OL］. arXiv preprint： 2112.08355， 2021.
[19]	VAN DER LENDE M， SABATELLI M， CARDENAS-CARTAGENA J. Interpretable function approximation with Gaussian processes in value-based model-free reinforcement learning［C］∥Northern Lights Deep Learning Conference 2025. New York： PMLR， 2025： 141-154.
[20]	CORSO A， MOSS R， KOREN M， et al. A survey of algorithms for black-box safety validation of cyber-physical systems［J］. Journal of Artificial Intelligence Research， 2022， 72： 377-428.
[21]	TABANDEH A， JIA G F， GARDONI P. A review and assessment of importance sampling methods for reliability analysis［J］. Structural Safety， 2022， 97： 102216.
[22]	ALI B S， OCHIENG W Y， MAJUMDAR A. ADS-B： Probabilistic safety assessment［J］. Journal of Navigation， 2017， 70（4）： 887-906.
[23]	GHATAS R， JACK D P， TSAKPINIS D， et al. Unmanned aircraft systems detect and avoid system： End-to-end verification and validation simulation study of minimum operational performance standards for integrating unmanned aircraft into the national airspace system［C］∥17th AIAA Aviation Technology， Integration， and Operations Conference. Reston： AIAA， 2017.
[24]	SCHULMAN J， WOLSKI F， DHARIWAL P， et al. Proximal policy optimization algorithms［DB/OL］. arXiv preprint： 1707.06347， 2017.
[25]	GENTON M G. Classes of kernels for machine learning： A statistics perspective［J］. Journal of Machine Learning Research， 2002， 2： 299-312.
[26]	KCOHENDERFER M J， WHEELER T A. Algorithms for optimization［M］. Cambridge： MIT Press， 2019： 1-20.
[27]	MURPHY K P. Machine learning： A probabilistic perspective［M］. Cambridge： MIT Press， 2012： 253-254
[28]	BISHOP， C M， NASRABADI N M. Pattern recognition and machine learning［M］. New York： springer， 2006： 365-366.
[29]	QUIÑONERO-CANDELA J， RASMUSSEN C E. A unifying view of sparse approximate Gaussian process regression［J］. Journal of Machine Learning Research， 2005， 6： 1939-1959.
[30]	JARUS Working Group 4. Detect and avoid-A working paper-Draft version 0.7［Z］. Joint Authorities for Rulemaking on Unmanned Systems （JARUS）， 2015.
[31]	TABASSUM A， SABATINI R， GARDI A. Probabilistic safety assessment for UAS separation assurance and collision avoidance systems［J］. Aerospace， 2019， 6（2）： 19.
[32]	RASCH M， UBBEN P T， MOST T， et al. Safety assessment and uncertainty quantification of automated driver assistance systems using stochastic analysis methods［C］∥NAFEMS World Congress 2019. Knutsford： NAFEMS， 2019： 16.

编号	故障树底事件	失效概率/每次飞行
1	ADS-B系统功能失效	1.0×10^-5
2	GPS系统数据丢失	2.0×10^-4
3	GPS系统信息误导	3.0×10^-5
4	RL模块提供错误“机动”	1.0×10^-5
5	RL模块硬件失效	3.5×10^-6
6	飞行控制功能失效	1.0×10^-6
7	无人机位于相撞航线

Safety assessment for airborne intelligent avoidance system based on Bayesian optimization

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 21

References 32

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Tao ZHANG, Pan LI, Zixu WANG, Zhenhua ZHU. Design of reward functions for helicopter attitude control in reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(S1): 732184-732184.
[2]	Kaifang WAN, Zhilin WU, Yunhui WU, Haozhi QIANG, Yibo WU, Bo LI. Cooperative location of multiple UAVs with deep reinforcement learning in GPS-denied environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331024-331024.
[3]	Lingfeng JIANG, Xinkai LI, Hai ZHANG, Hanwei LI, Hongli ZHANG. Mapless navigation of UAVs in dynamic environments based on an improved TD3 algorithm [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331035-331035.
[4]	Zan MA, Jie BAI, Yong CHEN, Ruihua LIU, Yanting ZHANG. Safety assessment for airborne CNN classifier based on conditional Gaussian PAC-Bayes [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(4): 330824-330824.
[5]	Min YANG, Guanjun LIU, Ziyuan ZHOU. Control of lunar landers based on secure reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630553-630553.
[6]	Qichao XIE, Chengyu CAO, Yiyun ZHAO, Fanbiao LI. Integrated guidance and control method based on deep reinforcement learning parameter tuning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(24): 632345-632345.
[7]	Jiachen LIU, Lei DONG, Zijing SUN, Ye NI, Xi CHEN, Peng WANG. An explainable decision-making method for resource allocation in IMA system based on PPO-SHAP [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(24): 331872-331872.
[8]	Tianqi FAN, Zhengxia ZOU, Zhenwei SHI. Typical remote sensing target detection with data synthesis based on reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(23): 631955-631955.
[9]	Chen WANG, Caisheng WEI, Zeyang YIN, Kai JIN, Xingchen LI. Collaborative planning of multi-UAV trajectories and communication strategies considering channel resource constraints [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 331837-331837.
[10]	Yizhe LUO, Hui ZHANG, Xinde YU, Zhao JIN, Shuo FENG, Yucheng SHI, Mingling XU. Hierarchical dynamic scheduling for multi-wave carrier-based aircraft ammunition support missions [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 331945-331945.
[11]	Xiangsong HUANG, Mengyu WANG, Dapeng PAN. Adversarial reinforcement learning-based UAV escape path planning method [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(17): 331637-331637.
[12]	Yu WANG, Zhipeng XIE, Yongjian TIAN, Guanglei MENG. Distributed UAV formation control with virtual structure guided reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(15): 331354-331354.
[13]	Wei CHEN, Lulu LI, Dong CHEN, Shaohui ZHANG, Yafei LI, Ke WANG, Yuanyuan JIN, Mingliang XU. Multi-aircraft cooperative decision-making methods driven by differentiated support demands for carrier-based aircraft [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531274-531274.
[14]	Xudong CHEN, Qiqi CHEN, Yizhe LUO, Jiabao WANG, Mingliang XU. Dynamic parallel scheduling of heterogeneous carrier-based aircraft deck support operations [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531329-531329.
[15]	Zheng WANG, Hua WANG, Keke CUI, Chaochao LI, Junnan LIU, Mingliang XU. Locally guided reinforcement learning for autonomous dispatching of carrier-based aircraft [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531333-531333.