ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Hierarchical dynamic scheduling for multi-wave carrier-based aircraft ammunition support missions
Received date: 2025-03-06
Revised date: 2025-03-19
Accepted date: 2025-05-13
Online published: 2025-06-06
Supported by
National Natural Science Foundation of China(62406292)
During the scheduling process of carrier-based aircraft ammunition support operations on aircraft carriers， the intricate interdependencies between various types of transfer equipment and support processes engender a highly non convex state space for the scheduling problem. Moreover， the substantial number of ammunition batches necessitating support further exacerbates the complexity by significantly expanding the search space， thereby diminishing the efficiency of the ammunition support process and impeding the ability to meet the dynamic real-time requirements of tasks. To address these challenges， this paper proposes a dynamic scheduling method for carrier-based aircraft ammunition support operations based on hierarchical reinforcement learning， inspired by the divide-and-conquer strategy. Initially， the scheduling decision process of ammunition support operations is decoupled and executed separately at the top and bottom levels， thereby alleviating the impact of the non-convexity and scale of the scheduling problem. Subsequently， decision network training for ammunition transfer equipment is conducted at the bottom level， and upon convergence， the trained model is integrated into the top-level environment to provide real-time feedback from the bottom level. Concurrently， at the top level， decision network training for ammunition support sequencing is performed， and a resource reservation mechanism is devised to recursively calculate ammunition transfer times， thereby determining the available time windows for transfer equipment and effectively circumventing conflicts in equipment usage. Ultimately， the proposed algorithm is validated in typical mission scenarios. The results indicate that， compared to traditional optimization algorithms， the proposed method substantially enhances decision-making real-time performance with only a minimal trade-off in scheduling time. It achieves a balanced trade-off between ammunition support time and the time required to generate support plans， rendering it well-suited for highly dynamic and strongly real-time support tasks.
Yizhe LUO , Hui ZHANG , Xinde YU , Zhao JIN , Shuo FENG , Yucheng SHI , Mingling XU . Hierarchical dynamic scheduling for multi-wave carrier-based aircraft ammunition support missions[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2025 , 46(18) : 331945 -331945 . DOI: 10.7527/S1000-6893.2025.31945
| [1] | 李亚飞， 吴庆顺， 徐明亮， 等. 基于强化学习的舰载机保障作业实时调度方法［J］. 中国科学： 信息科学， 2021， 51（2）： 247-262. |
| LI Y F， WU Q S， XU M L， et al. Real-time scheduling for carrier-borne aircraft support operations： A reinforcement learning approach［J］. Scientia Sinica （Informationis）， 2021， 51（2）： 247-262 （in Chinese）. | |
| [2] | GUO F， HAN W， SU X C， et al. A bi-population immune algorithm for weapon transportation support scheduling problem with pickup and delivery on aircraft carrier deck［J］. Defence Technology， 2023， 22（4）： 119-134. |
| [3] | 张少辉， 刘舜， 李亚飞， 等. 航空母舰舰载机弹药保障作业调度优化算法［J］. 航空学报， 2023， 44（20）： 228485. |
| ZHANG S H， LIU S， LI Y F， et al. Optimization algorithm for ammunition support operation scheduling of carrier-borne aircraft［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（20）： 228485 （in Chinese）. | |
| [4] | 高亮， 张国辉， 王晓娟. 柔性作业车间调度智能算法及其应用［M］. 武汉： 华中科技大学出版社， 2012. |
| GAO L， ZHANG G H， WANG X J. Intelligent algorithm for flexible job shop scheduling and its application［M］. Wuhan： Huazhong University of Science and Technology Press， 2012 （in Chinese）. | |
| [5] | 袁泉， 马羚， 吕晓峰. 母舰航空弹药转运流程规划方法［J］. 火力与指挥控制， 2024， 49（5）： 88-95， 101. |
| YUAN Q， MA L， LYU X F. Research on planning methods of aviation ammunition transfer process of aircraft carrier［J］. Fire Control & Command Control， 2024， 49（5）： 88-95， 101 （in Chinese）. | |
| [6] | 韩庆田， 曹文静， 苏涛. 基于遗传算法的舰载机保障流程研究［J］. 科学技术与工程， 2012， 12（35）： 9784-9787. |
| HAN Q T， CAO W J， SU T. Research on maintenance support schedule for carrier aircraft based on genetic algorithm［J］. Science Technology and Engineering， 2012， 12（35）： 9784-9787 （in Chinese）. | |
| [7] | 张洪亮， 刘建伟， 马羚， 等. 基于离散粒子群的舰载机弹药调度［J］. 舰船电子工程， 2021， 41（4）： 146-149. |
| ZHANG H L， LIU J W， MA L， et al. Ammunition scheduling of carrier based aircraft based on discrete particle swarm optimization［J］. Ship Electronic Engineering， 2021， 41（4）： 146-149 （in Chinese）. | |
| [8] | YUAN Q， WANG L， ZHENG X， et al. Ammunition scheduling of shipboard aircraft according to improved ant colony algorithm［C］∥Proceedings of the 2022 5th International Conference on Algorithms， Computing and Artificial Intelligence. New York： ACM， 2022： 1-7. |
| [9] | WANG L T， LI F Q， HUANG J R， et al. Optimization design of ammunition scheduling scheme for carrier-based aircraft based on improved DPSO algorithm［C］∥Proceedings of the 2022 5th International Conference on Algorithms， Computing and Artificial Intelligence. New York： ACM， 2022： 1-5. |
| [10] | ZHENG X M， LI B， WANG L T， et al. Design of carrier ammunition scheduling scheme based on improved genetic algorithm［C］∥Proceedings of the 7th International Conference on Control Engineering and Artificial Intelligence. New York： ACM， 2023： 162-167. |
| [11] | LIU M， LI G F. Ammunition scheduling method in air-borne weapon depot based on improved genetic algorithm［J］. Journal of Physics： Conference Series， 2021， 1948（1）： 012050. |
| [12] | 王丰， 李瑞鹏. 航母航空弹药保障能力优化的可拓策略生成研究［J］. 兵工自动化， 2023， 42（6）： 8-11， 26. |
| WANG F， LI R P. Research on extension strategy generation of aircraft carrier air ammunition support capability optimization［J］. Ordnance Industry Automation， 2023， 42（6）： 8-11， 26 （in Chinese）. | |
| [13] | 刘哲， 马俊飞， 陈佳峰， 等. 基于改进灰狼算法的舰载机弹药保障调度优化［J］. 系统工程与电子技术， 2024， 46（4）： 1264-1272. |
| LIU Z， MA J F， CHEN J F， et al. Carrier-based aircraft ammunition support scheduling optimization based on improved grey wolf optimizer algorithm［J］. Systems Engineering and Electronics， 2024， 46（4）： 1264-1272 （in Chinese）. | |
| [14] | 吕晓峰， 杨东泽， 马羚， 等. 基于改进遗传算法的舰载机弹药挂载调度［J］. 电光与控制， 2024， 31（1）： 82-86. |
| LYU X F， YANG D Z， MA L， et al. Carrier-based aircraft ammunition loading scheduling based on improved genetic algorithm［J］. Electronics Optics & Control， 2024， 31（1）： 82-86 （in Chinese）. | |
| [15] | 刘珏， 王能建， 罗旭， 等. 采用改进遗传算法的舰载机保障调度方法［J］. 国防科技大学学报， 2020， 42（2）： 194-205. |
| LIU J， WANG N J， LUO X， et al. Deck operation scheduling method of carrier-based aircraft based on improved genetic algorithm［J］. Journal of National University of Defense Technology， 2020， 42（2）： 194-205 （in Chinese）. | |
| [16] | 陶俊权， 苏析超， 韩维， 等. 基于EDA算法的航母弹药调度优化研究［J］. 兵器装备工程学报， 2022， 43（5）： 125-131. |
| TAO J Q， SU X C， HAN W， et al. Study of aircraft carrier ammunition scheduling optimization based on EDA algorithm［J］. Journal of Ordnance Equipment Engineering， 2022， 43（5）： 125-131 （in Chinese）. | |
| [17] | LIU Y J， HAN W， SU X C， et al. Optimization of fixed aviation support resource station configuration for aircraft carrier based on aircraft dispatch mission scheduling［J］. Chinese Journal of Aeronautics， 2023， 36（2）： 127-138. |
| [18] | KAYHAN B M， YILDIZ G. Reinforcement learning applications to machine scheduling problems： a comprehensive literature review［J］. Journal of Intelligent Manufacturing， 2023， 34（3）： 905-929. |
| [19] | 钟敬伟， 石宇强. 基于DQN的智能工厂作业车间调度［J］. 现代制造工程， 2021（9）： 17-23， 93. |
| ZHONG J W， SHI Y Q. Job shop scheduling based on DQN algorithm in intelligent factory［J］. Modern Manufacturing Engineering， 2021（9）： 17-23， 93 （in Chinese）. | |
| [20] | 王凌， 潘子肖. 基于深度强化学习与迭代贪婪的流水车间调度优化［J］. 控制与决策， 2021， 36（11）： 2609-2617. |
| WANG L， PAN Z X. Scheduling optimization for flow-shop based on deep reinforcement learning and iterative greedy method［J］. Control and Decision， 2021， 36（11）： 2609-2617 （in Chinese）. | |
| [21] | 李宝帅， 叶春明. 深度强化学习算法求解作业车间调度问题［J］. 计算机工程与应用， 2021， 57（23）： 248-254. |
| LI B S， YE C M. Job shop scheduling problem based on deep reinforcement learning［J］. Computer Engineering and Applications， 2021， 57（23）： 248-254 （in Chinese）. | |
| [22] | 白天， 罗永亮， 刘敬， 等. 基于变作业窗深度强化学习的舰面保障动态调度方法［J］. 船舶工程， 2021， 43（S2）： 117-123. |
| BAI T， LUO Y L， LIU J， et al. Dynamic aircraft scheduling method on flight deck based on variable operation window deep reinforcenemnt learning［J］. Ship Engineering， 2021， 43（S2）： 117-123 （in Chinese）. | |
| [23] | CHEN S Y， YU Y， DA Q， et al. Stabilizing reinforcement learning in dynamic environment with application to online recommendation［C］∥Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York：ACM，2018： 1187-1196. |
| [24] | 袁子龙， 何非， 赵建波， 等. 航母舰载机保障作业任务分配及弹药转运调度优化方法［J］. 兵工学报， 2025， 46（5）： 286-299. |
| YUAN Z L， HE F， ZHAO J B， et al. Optimization method of carrier-borne aircraft support operation assignment and ammunition transport scheduling［J］. Acta Armamentarii， 2025， 46（5）： 286-299 （in Chinese）. | |
| [25] | 吕晓峰， 杨东泽， 马羚. 舰载机模块化弹药调度方案优化设计［J］. 系统工程与电子技术， 2023， 45（2）： 465-471. |
| LYU X F， YANG D Z， MA L. Optimal design of modular ammunition scheduling scheme for carrier-based aircraft［J］. Systems Engineering and Electronics， 2023， 45（2）： 465-471 （in Chinese）. | |
| [26] | 郭漩， 王宁， 于淑彤， 等. 航空母舰多阶段弹药转运序列可视分析［J/OL］. 计算机辅助设计与图形学学报， （2024-07-25）［2025-03-06］. . |
| GUO X， WANG N， YU S T， et al. Visual analysis for multi-stage ammunition transfer sequences on aircraft carrier［J/OL］. Journal of Computer-Aided Design & Computer Graphics， （2024-07-25） ［2025-03-06］. （in Chinese）. | |
| [27] | 田德红， 何建敏， 齐洁， 等. 航空弹药动态调运决策优化建模与仿真研究［J］. 西北工业大学学报， 2018， 36（6）： 1236-1242. |
| TIAN D H， HE J M， QI J， et al. Research on the modeling and simulation of optimal dynamic aerial ammunition scheduling and transportation［J］. Journal of Northwestern Polytechnical University， 2018， 36（6）： 1236-1242 （in Chinese）. | |
| [28] | 刘哲， 陈佳峰， 马俊飞， 等. 舰载机弹药保障调度仿真系统［J］. 系统仿真学报， 2024， 36（7）： 1621-1630. |
| LIU Z， CHEN J F， MA J F， et al. Simulation system for carrier-based aircraft ammunition support scheduling［J］. Journal of System Simulation， 2024， 36（7）： 1621-1630 （in Chinese）. | |
| [29] | SCHULMAN J， WOLSKI F， DHARIWAL P， et al. Proximal policy optimization algorithms［DB/OL］. arXiv preprint： 1707. 06347， 2017. |
| [30] | BRITTAIN M， WEI P. Hierarchical reinforcement learning with deep nested agents［DB/OL］. arXiv preprint： 1805. 07008， 2018. |
/
| 〈 |
|
〉 |
CJA