面向舰载机多波次弹药保障任务的分层动态调度

doi:10.7527/S1000-6893.2025.31945

Abstract

Abstract:

During the scheduling process of carrier-based aircraft ammunition support operations on aircraft carriers， the intricate interdependencies between various types of transfer equipment and support processes engender a highly non convex state space for the scheduling problem. Moreover， the substantial number of ammunition batches necessitating support further exacerbates the complexity by significantly expanding the search space， thereby diminishing the efficiency of the ammunition support process and impeding the ability to meet the dynamic real-time requirements of tasks. To address these challenges， this paper proposes a dynamic scheduling method for carrier-based aircraft ammunition support operations based on hierarchical reinforcement learning， inspired by the divide-and-conquer strategy. Initially， the scheduling decision process of ammunition support operations is decoupled and executed separately at the top and bottom levels， thereby alleviating the impact of the non-convexity and scale of the scheduling problem. Subsequently， decision network training for ammunition transfer equipment is conducted at the bottom level， and upon convergence， the trained model is integrated into the top-level environment to provide real-time feedback from the bottom level. Concurrently， at the top level， decision network training for ammunition support sequencing is performed， and a resource reservation mechanism is devised to recursively calculate ammunition transfer times， thereby determining the available time windows for transfer equipment and effectively circumventing conflicts in equipment usage. Ultimately， the proposed algorithm is validated in typical mission scenarios. The results indicate that， compared to traditional optimization algorithms， the proposed method substantially enhances decision-making real-time performance with only a minimal trade-off in scheduling time. It achieves a balanced trade-off between ammunition support time and the time required to generate support plans， rendering it well-suited for highly dynamic and strongly real-time support tasks.

Key words: hierarchical reinforcement learning, carrier-based aircraft, scheduling optimization, resource constraints, ammunition support operations

CLC Number:

Yizhe LUO, Hui ZHANG, Xinde YU, Zhao JIN, Shuo FENG, Yucheng SHI, Mingling XU. Hierarchical dynamic scheduling for multi-wave carrier-based aircraft ammunition support missions[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 331945.

Figures/Tables 25

Fig.1

Fig.2

Fig.3

Table 1

Status Parameter

状态	含义
$P t$	弹药所处位置状态
$C t$	运载车辆占用状态
$D t$	设备占用状态
$D t 1$	弹药库占用状态
$D t 2$	上层升降机占用状态
$D t 3$	弹药装配区占用状态
$D t 4$	下层升降机占用状态
$D t 5$	阵位占用状态

Table 1

Fig.4

Fig.5

Fig.6

Fig.7

Table 2

Table 3

Fig.8

Fig.9

Fig.10

Table 4

Fig.11

Table 5

Fig.12

Table 6

Fig.13

Table 7

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

References 30

[1]	李亚飞，吴庆顺，徐明亮，等. 基于强化学习的舰载机保障作业实时调度方法［J］. 中国科学：信息科学， 2021， 51（2）： 247-262.
	LI Y F， WU Q S， XU M L， et al. Real-time scheduling for carrier-borne aircraft support operations： A reinforcement learning approach［J］. Scientia Sinica （Informationis）， 2021， 51（2）： 247-262 （in Chinese）.
[2]	GUO F， HAN W， SU X C， et al. A bi-population immune algorithm for weapon transportation support scheduling problem with pickup and delivery on aircraft carrier deck［J］. Defence Technology， 2023， 22（4）： 119-134.
[3]	张少辉，刘舜，李亚飞，等. 航空母舰舰载机弹药保障作业调度优化算法［J］. 航空学报， 2023， 44（20）： 228485.
	ZHANG S H， LIU S， LI Y F， et al. Optimization algorithm for ammunition support operation scheduling of carrier-borne aircraft［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（20）： 228485 （in Chinese）.
[4]	高亮，张国辉，王晓娟. 柔性作业车间调度智能算法及其应用［M］. 武汉：华中科技大学出版社， 2012.
	GAO L， ZHANG G H， WANG X J. Intelligent algorithm for flexible job shop scheduling and its application［M］. Wuhan： Huazhong University of Science and Technology Press， 2012 （in Chinese）.
[5]	袁泉，马羚，吕晓峰. 母舰航空弹药转运流程规划方法［J］. 火力与指挥控制， 2024， 49（5）： 88-95， 101.
	YUAN Q， MA L， LYU X F. Research on planning methods of aviation ammunition transfer process of aircraft carrier［J］. Fire Control & Command Control， 2024， 49（5）： 88-95， 101 （in Chinese）.
[6]	韩庆田，曹文静，苏涛. 基于遗传算法的舰载机保障流程研究［J］. 科学技术与工程， 2012， 12（35）： 9784-9787.
	HAN Q T， CAO W J， SU T. Research on maintenance support schedule for carrier aircraft based on genetic algorithm［J］. Science Technology and Engineering， 2012， 12（35）： 9784-9787 （in Chinese）.
[7]	张洪亮，刘建伟，马羚，等. 基于离散粒子群的舰载机弹药调度［J］. 舰船电子工程， 2021， 41（4）： 146-149.
	ZHANG H L， LIU J W， MA L， et al. Ammunition scheduling of carrier based aircraft based on discrete particle swarm optimization［J］. Ship Electronic Engineering， 2021， 41（4）： 146-149 （in Chinese）.
[8]	YUAN Q， WANG L， ZHENG X， et al. Ammunition scheduling of shipboard aircraft according to improved ant colony algorithm［C］∥Proceedings of the 2022 5th International Conference on Algorithms， Computing and Artificial Intelligence. New York： ACM， 2022： 1-7.
[9]	WANG L T， LI F Q， HUANG J R， et al. Optimization design of ammunition scheduling scheme for carrier-based aircraft based on improved DPSO algorithm［C］∥Proceedings of the 2022 5th International Conference on Algorithms， Computing and Artificial Intelligence. New York： ACM， 2022： 1-5.
[10]	ZHENG X M， LI B， WANG L T， et al. Design of carrier ammunition scheduling scheme based on improved genetic algorithm［C］∥Proceedings of the 7th International Conference on Control Engineering and Artificial Intelligence. New York： ACM， 2023： 162-167.
[11]	LIU M， LI G F. Ammunition scheduling method in air-borne weapon depot based on improved genetic algorithm［J］. Journal of Physics： Conference Series， 2021， 1948（1）： 012050.
[12]	王丰，李瑞鹏. 航母航空弹药保障能力优化的可拓策略生成研究［J］. 兵工自动化， 2023， 42（6）： 8-11， 26.
	WANG F， LI R P. Research on extension strategy generation of aircraft carrier air ammunition support capability optimization［J］. Ordnance Industry Automation， 2023， 42（6）： 8-11， 26 （in Chinese）.
[13]	刘哲，马俊飞，陈佳峰，等. 基于改进灰狼算法的舰载机弹药保障调度优化［J］. 系统工程与电子技术， 2024， 46（4）： 1264-1272.
	LIU Z， MA J F， CHEN J F， et al. Carrier-based aircraft ammunition support scheduling optimization based on improved grey wolf optimizer algorithm［J］. Systems Engineering and Electronics， 2024， 46（4）： 1264-1272 （in Chinese）.
[14]	吕晓峰，杨东泽，马羚，等. 基于改进遗传算法的舰载机弹药挂载调度［J］. 电光与控制， 2024， 31（1）： 82-86.
	LYU X F， YANG D Z， MA L， et al. Carrier-based aircraft ammunition loading scheduling based on improved genetic algorithm［J］. Electronics Optics & Control， 2024， 31（1）： 82-86 （in Chinese）.
[15]	刘珏，王能建，罗旭，等. 采用改进遗传算法的舰载机保障调度方法［J］. 国防科技大学学报， 2020， 42（2）： 194-205.
	LIU J， WANG N J， LUO X， et al. Deck operation scheduling method of carrier-based aircraft based on improved genetic algorithm［J］. Journal of National University of Defense Technology， 2020， 42（2）： 194-205 （in Chinese）.
[16]	陶俊权，苏析超，韩维，等. 基于EDA算法的航母弹药调度优化研究［J］. 兵器装备工程学报， 2022， 43（5）： 125-131.
	TAO J Q， SU X C， HAN W， et al. Study of aircraft carrier ammunition scheduling optimization based on EDA algorithm［J］. Journal of Ordnance Equipment Engineering， 2022， 43（5）： 125-131 （in Chinese）.
[17]	LIU Y J， HAN W， SU X C， et al. Optimization of fixed aviation support resource station configuration for aircraft carrier based on aircraft dispatch mission scheduling［J］. Chinese Journal of Aeronautics， 2023， 36（2）： 127-138.
[18]	KAYHAN B M， YILDIZ G. Reinforcement learning applications to machine scheduling problems： a comprehensive literature review［J］. Journal of Intelligent Manufacturing， 2023， 34（3）： 905-929.
[19]	钟敬伟，石宇强. 基于DQN的智能工厂作业车间调度［J］. 现代制造工程， 2021（9）： 17-23， 93.
	ZHONG J W， SHI Y Q. Job shop scheduling based on DQN algorithm in intelligent factory［J］. Modern Manufacturing Engineering， 2021（9）： 17-23， 93 （in Chinese）.
[20]	王凌，潘子肖. 基于深度强化学习与迭代贪婪的流水车间调度优化［J］. 控制与决策， 2021， 36（11）： 2609-2617.
	WANG L， PAN Z X. Scheduling optimization for flow-shop based on deep reinforcement learning and iterative greedy method［J］. Control and Decision， 2021， 36（11）： 2609-2617 （in Chinese）.
[21]	李宝帅，叶春明. 深度强化学习算法求解作业车间调度问题［J］. 计算机工程与应用， 2021， 57（23）： 248-254.
	LI B S， YE C M. Job shop scheduling problem based on deep reinforcement learning［J］. Computer Engineering and Applications， 2021， 57（23）： 248-254 （in Chinese）.
[22]	白天，罗永亮，刘敬，等. 基于变作业窗深度强化学习的舰面保障动态调度方法［J］. 船舶工程， 2021， 43（S2）： 117-123.
	BAI T， LUO Y L， LIU J， et al. Dynamic aircraft scheduling method on flight deck based on variable operation window deep reinforcenemnt learning［J］. Ship Engineering， 2021， 43（S2）： 117-123 （in Chinese）.
[23]	CHEN S Y， YU Y， DA Q， et al. Stabilizing reinforcement learning in dynamic environment with application to online recommendation［C］∥Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York：ACM，2018： 1187-1196.
[24]	袁子龙，何非，赵建波，等. 航母舰载机保障作业任务分配及弹药转运调度优化方法［J］. 兵工学报， 2025， 46（5）： 286-299.
	YUAN Z L， HE F， ZHAO J B， et al. Optimization method of carrier-borne aircraft support operation assignment and ammunition transport scheduling［J］. Acta Armamentarii， 2025， 46（5）： 286-299 （in Chinese）.
[25]	吕晓峰，杨东泽，马羚. 舰载机模块化弹药调度方案优化设计［J］. 系统工程与电子技术， 2023， 45（2）： 465-471.
	LYU X F， YANG D Z， MA L. Optimal design of modular ammunition scheduling scheme for carrier-based aircraft［J］. Systems Engineering and Electronics， 2023， 45（2）： 465-471 （in Chinese）.
[26]	郭漩，王宁，于淑彤，等. 航空母舰多阶段弹药转运序列可视分析［J/OL］. 计算机辅助设计与图形学学报，（2024-07-25）［2025-03-06］. .
	GUO X， WANG N， YU S T， et al. Visual analysis for multi-stage ammunition transfer sequences on aircraft carrier［J/OL］. Journal of Computer-Aided Design & Computer Graphics，（2024-07-25）［2025-03-06］. （in Chinese）.
[27]	田德红，何建敏，齐洁，等. 航空弹药动态调运决策优化建模与仿真研究［J］. 西北工业大学学报， 2018， 36（6）： 1236-1242.
	TIAN D H， HE J M， QI J， et al. Research on the modeling and simulation of optimal dynamic aerial ammunition scheduling and transportation［J］. Journal of Northwestern Polytechnical University， 2018， 36（6）： 1236-1242 （in Chinese）.
[28]	刘哲，陈佳峰，马俊飞，等. 舰载机弹药保障调度仿真系统［J］. 系统仿真学报， 2024， 36（7）： 1621-1630.
	LIU Z， CHEN J F， MA J F， et al. Simulation system for carrier-based aircraft ammunition support scheduling［J］. Journal of System Simulation， 2024， 36（7）： 1621-1630 （in Chinese）.
[29]	SCHULMAN J， WOLSKI F， DHARIWAL P， et al. Proximal policy optimization algorithms［DB/OL］. arXiv preprint： 1707. 06347， 2017.
[30]	BRITTAIN M， WEI P. Hierarchical reinforcement learning with deep nested agents［DB/OL］. arXiv preprint： 1805. 07008， 2018.

参数	数值	参数	数值
经验库容量	4 096	时间步数	1×10⁷
学习率	可变参数	激活函数	Tanh
批大小	128	剪切参数	0.4
折扣率	可变参数	值函数系数	0.5

弹药类型	设备使用时长/min
弹药类型	l1	l2	l3	l4	l5
1	1	3	4	3	5
2	1	3	4	3	5
3	1	3	4	3	5
4	1	3	4	3	5
5	1	3	4	3	5

Hierarchical dynamic scheduling for multi-wave carrier-based aircraft ammunition support missions

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 25

References 30

Related Articles 15

Recommended Articles

Metrics

Comments

弹药数	弹药转运时长/min
弹药数	本文算法	GA	PSO	Greedy
5	32	32	32	32
6	35	36	36	38
7	44	43	43	45
8	47	47	47	53
9	56	57	56	61
10	58	60	60	68

[1]	Xiaochen LYU, Jingping SHI, Yongxi LYU, Gengnong LI. Flow angle reconstruction algorithm for MAGIC CARPET landing with sensor failure [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531159-531159.
[2]	Fang GUO, Wei HAN, Yujie LIU, Jie LIU, Xichao SU, Liangliang CHENG. Scheduling for maintenance and service support of carrier-based aircraft based on variable operation process [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531195-531195.
[3]	Yu LI, Tongwen CHEN, Zhigang WANG, Chiyung WEN, Xiaoxiong LIU. Incremental control of direct lift landing based on predefined-time theory [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531163-531163.
[4]	Wei CHEN, Lulu LI, Dong CHEN, Shaohui ZHANG, Yafei LI, Ke WANG, Yuanyuan JIN, Mingliang XU. Multi-aircraft cooperative decision-making methods driven by differentiated support demands for carrier-based aircraft [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531274-531274.
[5]	Xudong CHEN, Qiqi CHEN, Yizhe LUO, Jiabao WANG, Mingliang XU. Dynamic parallel scheduling of heterogeneous carrier-based aircraft deck support operations [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531329-531329.
[6]	Zheng WANG, Hua WANG, Keke CUI, Chaochao LI, Junnan LIU, Mingliang XU. Locally guided reinforcement learning for autonomous dispatching of carrier-based aircraft [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531333-531333.
[7]	Xinze XU, Guanxin HONG, Liang DU, Gang LIU. Manual approach and landing model of carrier-based aircraft in complex environments [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 531802-531802.
[8]	Jiaxing WANG, Hao CHEN, Zheng SHAO, Yang ZHANG. Direct lift landing control method based on flight path angle command [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 532162-532162.
[9]	Zhibing ZHANG, Ziyang ZHEN. Research progress on guidance and control of fixed-wing manned and unmanned carrier-based aircraft landing [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(13): 532336-532336.
[10]	Guang LIU, Hua WANG, Youfang LIN, Shuo HE, Yafei LI, Mingliang XU. Adaptive batch matching decision method for carrier-based aircraft support operations [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(1): 330615-330615.
[11]	Dapeng ZHOU, Xiaolei QU. Knowledge-based intelligent pigeon-inspired optimization of carrier-based aircraft landing control [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(S1): 730801-730801.
[12]	FU Jiawei, YU Jialong, LIU Chao, WANG Muguo, WANG Zizi. Key technologies of aerodynamics design of stealth carrier-based aircraft [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(8): 525804-525804.
[13]	WANG Xiao, CHENG Jianhui, SHEN Tianrong, XU Baocheng, MENG Xuan. Numerical simulation research of influence of wake flow field on inlet temperature field of carrier-based aircraft [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(8): 525795-525795.
[14]	CHEN Yueliang, CHEN Liang, BIAN Guixue, YANG Xiangning, GUAN Yu, ZHANG Yong, HE Gang. Corrosion protection control and calendar life design of advanced carrier-based aircraft [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(8): 525786-525786.
[15]	LIU Dong, WU Jiaren, ZHOU Yizhou, LIU Zhenxiang, LI Yu, WANG Mingze. Practice and prospects of comprehensive support technologies of carrier-based aircraft [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(8): 525802-525802.