During the scheduling process of carrier-based aircraft ammunition support operations on aircraft carriers, the intricate interdependencies between various types of transfer equipment and support processes engender a highly non convex state space for the scheduling problem. Moreover, the substantial number of ammunition batches necessitating support further exacerbates the complexity by significantly expanding the search space, thereby diminishing the efficiency of the ammunition support process and impeding the ability to meet the dynamic real-time requirements of tasks.To address these challenges, this paper proposes a dynamic scheduling method for carrier-based aircraft ammunition support operations, which is founded on hierarchical reinforcement learning and inspired by the divide-and-conquer strategy. Initially, the scheduling decision process of ammunition support operations is decoupled and executed separately at the top and bottom levels, thereby alleviating the impact of the non-convexity and scale of the scheduling problem. Subsequently, decision network training for ammunition transfer equipment is conducted at the bottom level, and upon convergence, the trained model is integrated into the top-level environment to provide real-time feedback from the bottom level. Concurrently, at the top level, decision network training for ammunition support sequencing is performed, and a resource reservation mechanism is devised to recursively calculate ammunition transfer times, thereby determining the available time windows for transfer equipment and effectively circumventing conflicts in equipment usage. Ultimately, the proposed algorithm is validated in typical mission scenarios. The results indicate that, compared to traditional optimization algorithms, the proposed method substantially enhances decision-making real-time performance with only a minimal trade-off in scheduling time. It achieves a balanced trade-off between ammunition support time and the time required to generate support plans, rendering it well-suited for highly dynamic and strongly real-time support tasks.
[1]李亚飞, 吴庆顺, 徐明亮, 等.基于强化学习的舰载机保障作业实时调度方法[J].中国科学信息科学, 2021, 51(02):247-261
[2]Guo F, Han W, Su X, et al.A Bi-population Immune Algorithm for Weapon Transportation Support Scheduling Problem with Pickup and Delivery on Aircraft Carrier Deck[J].Defence Technology, 2023, 22(04):119-134
[3]张少辉, 刘舜, 李亚飞, 等.航空母舰舰载机弹药保障作业调度优化算法[J].航空学报, 2023, 44(20):230-247
[4]高亮, 张国辉, 王晓娟（Gao L, Zhang G H, Wang X J）.柔性作业车间调度智能算法及其应用（Intelligent Algorithm for Flexible Job Shop Scheduling and its Application）[M]. 武汉: 华中科技大学出版社（Huazhong University of Science and Technology Press）, 2012.
[5]袁泉, 马羚, 吕晓峰.母舰航空弹药转运流程规划方法[J].火力与指挥控制, 2024, 49(05):88-95
[6]韩庆田, 曹文静, 苏涛.基于遗传算法的舰载机保障流程研究[J].科学技术与工程, 2012, 12(35):9784-9787
[7]张洪亮, 刘建伟, 马羚, 等.基于离散粒子群的舰载机弹药调度[J].舰船电子工程, 2021, 41(04):146-149
[8]Yuan Q, Wang L, Zheng X, et al.Ammunition Scheduling of Shipboard Aircraft According to Improved Ant Colony Algorithm[C]//Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence. 2022: 1-7.
[9]Wang L, Li F, Huang J, et al.Optimization Design of Ammunition Scheduling Scheme for Carrier-based Aircraft based on Improved DPSO Algorithm[C]//Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence. 2022: 1-5.
[10]Zheng X, Li B, Wang L, et al.Design of Carrier Ammunition Scheduling Scheme Based on Improved Genetic Algorithm[C]//Proceedings of the 7th International Conference on Control Engineering and Artificial Intelligence.
[11] Liu M, Li G.Ammunition Scheduling Method in Airborne Weapon Depot Based on Improved Genetic Algorithm[C]//Journal of Physics: Conference Series. IOP Publishing, 2021, 1948(1): 012050.
[12]王丰, 李瑞鹏.航母航空弹药保障能力优化的可拓策略生成研究[J].兵工自动化, 2023, 42(06):8-11
[13]刘哲, 马俊飞, 陈佳峰, 等.基于改进灰狼算法的舰载机弹药保障调度优化[J].系统工程与电子技术, 2024, 46(4):1264-1272
[14]吕晓峰, 杨东泽, 马羚.基于改进遗传算法的舰载机弹药挂载调度[J].电光与控制, 2024, 31(01):82-86
[15]刘珏, 王能建, 罗旭, 等.采用改进遗传算法的舰载机保障调度方法[J].国防科技大学学报, 2020, 42(02):194-205
[16]陶俊权, 苏析超, 韩维, 等.基于 算法的航母弹药调度优化研究[J].兵器装备工程学报, 2022, 43(05):125-131
[17]Liu Y J, Han W, Su X C, et al.Optimization of fixed aviation support resource station configuration for aircraft carrier based on aircraft dispatch mission scheduling[J].Chinese Journal of Aeronautics, 2023, 36(2):127-138
[18]KAYHAN B M, YILDIZ G.Reinforcement Learning Applications to Machine Scheduling Problems: A Comprehensive Literature Review[J].Journal of Intelligent Manufacturing, 2023, 34(3):905-929
[19]钟敬伟, 石宇强.基于 的智能工厂作业车间调度[J].现代制造工程, 2021, 44(09):17-23
[20]王凌, 潘子肖.基于深度强化学习与迭代贪婪的流水车间调度优化[J].控制与决策, 2021, 36(11):2609-2617
[21]李宝帅, 叶春明.深度强化学习算法求解作业车间调度问题[J].计算机工程与应用, 2021, 57(23):248-254
[22]白天, 罗永亮, 刘敬, 等.基于变作业窗深度强化学习的舰面保障动态调度方法[J].船舶工程, 2021, 43(S2):117-123
[23]Chen S Y, Yu Y, Da Q, et al.Stabilizing reinforcement learning in dynamic environment with application to online recommendation[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining. 2018: 1187-1196.
[24]袁子龙, 何非, 赵建波, 等.航母舰载机保障作业任务分配及弹药转运调度优化方法[J].兵工学报, 2024, 46(10):1-16
[25]吕晓峰, 杨东泽, 马羚.舰载机模块化弹药调度方案优化设计[J].系统工程与电子技术, 2023, 45(02):465-471
[26]郭漩, 王宁, 于淑彤, 等.航空母舰多阶段弹药转运序列可视分析[J].计算机辅助设计与图形学学报, 2025, 36(02):1-9
[27]田德红, 何建敏, 齐洁, 等.航空弹药动态调运决策优化建模与仿真研究[J].西北工业大学学报, 2018, 36(06):1236-1242
[28]刘哲, 陈佳峰, 马俊飞, 等.舰载机弹药保障调度仿真系统[J].系统仿真学报, 2024, 36(07):1621-1630
[29]Schulman J, Wolski F, Dhariwal P, et al.Proximal policy optimization algorithms[J].[J].arXiv prep.[J].rXiv:1707.06347, 2017, :-
[30]Brittain M, Wei P.Hierarchical Reinforcement Learning with Deep Nested Agents[J].[J].arXiv prep.[J].rXiv:1805.07008, 2018, :-
[31]徐嘉琦, 田野.基于改进遗传算法的柔性流水车间调度研究[J].制造技术与机床, 2024, 74(04):181-187
CJA