基于舰面环境模型的舰载机保障作业调度算法

doi:10.7527/S1000-6893.2026.33180

Abstract

Abstract:

In aircraft carrier flight deck scheduling operations， Model-Free Reinforcement Learning （MFRL） is constrained by the precision of physical environment modeling under dynamic deck conditions. In contrast， Model-Based Reinforcement Learning （MBRL） faces challenges of high computational complexity and convergence difficulties due to the interdependent co-optimization between the environment model and the decision-making model during iterative training. To address these issues， This paper proposes a hybrid reinforcement learning framework that integrates Model-Based and Model-Free （MB-MF） characteristics. First， a deep neural network-based deck environment model is trained using historical scheduling data to accurately predict state transitions within a minimal tolerance range. Then， the converged environment model is embedded into the interactive environment in place of the real environment， where a scheduling agent is trained using the Deep Q-Network （DQN） algorithm， thereby decoupling environment model learning from policy optimization. Experimental results demonstrate that， compared to MFRL using the physical environment， the proposed method achieves a performance gap of only 4% without requiring precise modeling. Moreover， it reduces the aircraft sortie time by 34% relative to the MBRL baseline. In resource-constrained scenarios， the decision-making speed is nearly 300 times faster than that of heuristic methods， while the scheduling quality is reduced by only 17%.

Key words: carrier-based aircraft, deck operations, complex job scheduling, reinforcement learning, deep learning

CLC Number:

V211.3

Yizhe LUO, Jiabao WANG, Xinde YU, Xudong CHEN, Zhao JIN, Shuo FENG, Yucheng SHI, Mingliang XU. Scheduling algorithm for carrier-based aircraft support operations based on deck environment model[J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(17): 333180.

Figures/Tables 20

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Table 1

Algorithm parameters preset

参数	含义	数值
$D M$	环境模型训练数据集	6×10⁵
$B M$	训练批量	32
$N M$	环境模型训练轮数	50
$E M$	状态转移结果可容忍误差	5×10^-2
$L M$	环境模型训练初始学习率	1×10^-3
$L R L$	强化学习训练初始学习率	1×10^-4
$N R L$	强化学习总步数	1×10⁶
$γ$	折扣因子	0.995
$λ 1$	作业完成奖励系数	1
$λ 2$	等待惩罚系数	-5
$λ 3$	调运惩罚系数	-10

Table 1

Fig.8

Table 2

Analysis of scheduling success rate

算法	$λ 1, λ 2, λ 3$ 取值	场景1调度成功率/%	场景2调度成功率/%
DQN	1，-5，-10	97.8	95.3
	1，-5，-5	97.1	93.4
	1，-10，-20	97.4	94.6
MB-MF	1，-5，-10	94.6	91.3
	1，-5，-5	93.3	89.7
	1，-10，-20	94.2	90.4
PPO	1，-5，-10	97.4	95.7
GA		100.00	100.00

Table 2

Table 3

Fig.9

Table 4

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

References 46

[1]	DIETZ D C， JENKINS R C. Analysis of aircraft sortie generation with the use of a fork-join queueing network model［J］. Naval Research Logistics， 1997， 44（2）： 153-164.
[2]	LIU Y J， HAN W， SU X C， et al. Optimization of fixed aviation support resource station configuration for aircraft carrier based on aircraft dispatch mission scheduling［J］. Chinese Journal of Aeronautics， 2023， 36（2）： 127-138.
[3]	WANG X， LIU J， SU X， et al. A review on carrier aircraft dispatch path planning and control on deck［J］. Chinese Journal of Aeronautics， 2020， 33（12）： 3039-3057.
[4]	刘翱，刘克. 舰载机保障作业调度问题研究进展［J］. 系统工程理论与实践， 2017， 37（1）： 49-60.
	LIU A， LIU K. Research progress on carrier-based aircraft support operation scheduling problem［J］. Systems Engineering-Theory & Practice， 2017， 37（1）： 49-60 （in Chinese）.
[5]	HARTMANN S， BRISKORN D. An updated survey of variants and extensions of the resource-constrained project scheduling problem［J］. European Journal of Operational Research， 2022， 297（1）： 1-14.
[6]	KHATAB A， DIALLO C， VENKATADRI U， et al. Optimization of the joint selective maintenance and repairperson assignment problem under imperfect maintenance［J］. Computers & Industrial Engineering， 2018， 125： 413-422.
[7]	陈小飞，时立攀，毕玉泉. 美军航母舰载机出动回收能力和飞行甲板控制策略探讨［J］. 舰船科学技术， 2020， 42（21）： 174-179.
	CHEN X F， SHI L P， BI Y Q. Discussion on launch and recovery capability and flight deck control strategy of US aircraft carrier［J］. Ship Science and Technology， 2020， 42（21）： 174-179 （in Chinese）.
[8]	刘洪铭，曾鸿雁，周伟，等. 基于改进粒子群算法作业车间调度问题的优化［J］. 山东大学学报（工学版）， 2019， 49（1）： 75-82.
	LIU H M， ZENG H Y， ZHOU W， et al. Optimization of job shop scheduling problem based on improved particle swarm algorithm［J］. Journal of Shandong University （Engineering Science）， 2019， 49（1）： 75-82 （in Chinese）.
[9]	ELTOUKHY A E E， WANG Z X， CHAN F T S， et al. Joint optimization using a leader-follower Stackelberg game for coordinated configuration of stochastic operational aircraft maintenance routing and maintenance staffing［J］. Computers & Industrial Engineering， 2018， 125： 46-68.
[10]	WU J， DAI M Q， WANG J J， et al. Carrier-based aircraft operation support scheduling based on apprenticeship learning algorithm［J］. Chinese Journal of Ship Research， 2022， 17（4）： 145-154.
[11]	LI M， YU M. Carrier-based aircraft support operation scheduling based on improved tabu search algorithm［J］. Chinese Journal of Ship Research， 2018， 13（5）： 132-138.
[12]	LI M W， QIN N， ZHU T， et al. Carrier aircraft scheduling optimization based on A* algorithm and genetic algorithm［C］∥ 2022 IEEE 11th Data Driven Control and Learning Systems Conference （DDCLS）. Piscataway： IEEE Press， 2022.
[13]	FU X Y， CHAN F T S， NIU B， et al. A multi-objective pigeon inspired optimization algorithm for fuzzy production scheduling problem considering mould maintenance［J］. Science China Information Sciences， 2019， 62（7）： 70202.
[14]	ZHANG W， DIETTERICH T G. A reinforcement learning approach to job-shop scheduling［C］∥ Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence （IJCAI）. San Francisco： Morgan Kaufmann， 1995.
[15]	YAO L W， CHEN Q C， GONG L H， et al. Multi-NPDQ： a multi-agent approach through deep reinforcement learning for operation scheduling［C］∥ International Conference on Intelligent Computing. Singapore： Springer， 2024.
[16]	HAO H J， ZHANG X Q， CHI Y， et al. Cooperative carrier aircraft support operation scheduling via multi-agent reinforcement learning［C］∥ 2023 24th IEEE International Conference on Mobile Data Management （MDM）. Piscataway： IEEE Press， 2023.
[17]	YAN Q， WANG H F. Double-layer Q-learning-based joint decision-making of dual resource-constrained aircraft assembly scheduling and flexible preventive maintenance［J］. IEEE Transactions on Aerospace and Electronic Systems， 2022， 58（6）： 4938-4952.
[18]	ÇALIŞIR S， PEHLIVANOĞLU M K. Model-free reinforcement learning algorithms： A survey［C］∥ 2019 27th Signal Processing and Communications Applications Conference （SIU）. Piscataway： IEEE Press， 2019.
[19]	ARULKUMARAN K， DEISENROTH M P， BRUNDA-GE M， et al. Deep reinforcement learning： A brief survey［J］. IEEE Signal Processing Magazine， 2017， 34（6）： 26-38.
[20]	RAVISHANKAR N R， VIJAYAKUMAR M V. Rein-forcement learning algorithms： Survey and classification［J］. Indian Journal of Science and Technology. 2017， 10（1）： 1-8.
[21]	WU P， ESCONTRELA A， HAFNER D， et al. Day-dreamer： world models for physical robot learning［C］∥ Conference on Robot Learning. Cambridge： PMLR， 2023.
[22]	PAL C V， LEON F. Brief survey of model-based reinforcement learning techniques［C］∥ 2020 24th International Conference on System Theory， Control and Computing （ICSTCC）. Piscataway： IEEE Press， 2020.
[23]	PLAAT A， KOSTERS W， PREUSS M. High-accuracy model-based reinforcement learning， a survey［J］. Artificial Intelligence Review， 2023， 56（9）： 9541-9573.
[24]	MOERLAND T M， BROEKENS J， PLAAT A， et al. Model-based reinforcement learning： A survey［J］. Foundations and Trends in Machine Learning， 2023， 16（1）： 1-118.
[25]	OSINSKI B， FINN C， ERHAN D， et al. Model-based reinforcement learning for atari［C］∥ International Conference on Learning Representations （ICLR）. Amberst： OpenReview， 2020.
[26]	閤泰梓，唐秋华，成丽新. 基于 DQN 协同进化算法的柔性作业车间能效调度优化［J］. 计算机集成制造系统， 2025， 31（2）： 411-422.
	GE T Z， TANG Q H， CHENG L X. Energy-efficient scheduling optimization for flexible job shop based on DQN co-evolutionary algorithm［J］. Computer Integrated Manufacturing Systems， 2025， 31（2）： 411-422 （in Chinese）.
[27]	MNIH V， KAVUKCUOGLU K， SILVER D， et al. Human-level control through deep reinforcement learning［J］. Nature， 2015， 518（7540）： 529-533.
[28]	RYAN J C， BANERJEE A G， CUMMINGS M L， et al. Comparing the performance of expert user heuristics and an integer linear program in aircraft carrier deck operations［J］. IEEE Transactions on Cybernetics， 2014， 44（6）： 761-773.
[29]	LI C J， SU X C， ZHANG Y， et al. Integrated scheduling method for fleet wave sorties and maintenance of naval distributed platforms［J］. Advanced Engineering Informatics， 2024， 59： 102340.
[30]	王佳怡，潘瑞林，秦飞. 改进遗传算法求解柔性作业车间调度问题［J］. 制造业自动化， 2022， 44（12）： 91-94， 106.
	WANG J Y， PAN R L， QIN F. Improved genetic algorithm for flexible job shop scheduling problem［J］. Manufacturing Automation， 2022， 44（12）： 91-94， 106 （in Chinese）.
[31]	JIANG T T， SU X C， HAN W. Optimization of support scheduling on deck of carrier aircraft based on improved differential evolution algorithm［C］∥ 2017 3rd IEEE International Conference on Control Science and Systems Engineering （ICCSSE）. Piscataway： IEEE Press， 2017.
[32]	CUI R W， HAN W， SU X C， et al. A multi-objective hyper-heuristic framework for integrated optimization of carrier-based aircraft flight deck operations scheduling and resource configuration［J］. Aerospace Science and Technology， 2020， 107： 106346.
[33]	TANG C J， ZENG W， ZHAO Z X. Deep reinforcement learning for solving aircraft carrier flight deck tasks scheduling problems［C］∥ Proceedings of the 2024 Guangdong-Hong Kong-Macao Greater Bay Area International Conference on Digital Economy and Artificial Intelligence. New York： ACM， 2024.
[34]	LIU R K， PIPLANI R， TORO C. Deep reinforcement learning for dynamic scheduling of a flexible job shop［J］. International Journal of Production Research， 2022， 60（13）： 4049-4069.
[35]	LI Y F， WU Q S， XU M L， et al. Real-time scheduling for carrier-borne aircraft support operations： A reinforcement learning approach［J］. Scientia Sinca Informationis， 2021， 51（2）： 247.
[36]	HAN W， LI C J， SU X C， et al. Real-time decision support for bolter recovery safety： Long short-term memory network-driven aircraft sequencing［J］. Defence Technology， 2026， 56： 184-205.
[37]	LI C J， HAN W， LI H X， et al. Deep reinforcement learning for carrier-based aircraft flight deck operations scheduling problem［J］. Neural Networks， 2026， 200： 108776.
[38]	LI C J， HAN W， ZHANG Y， et al. End-to-end scheduling for carrier-based aircraft sortie operations using deep reinforcement learning［J］. Information Processing & Management， 2026， 63（4）： 104590.
[39]	LI C J， ZHANG Y， HAN W， et al. Real-time sequencing problem for aircraft recovery on carriers： A safe deep reinforcement learning approach［J］. Chinese Journal of Aeronautics， 2025： 103911.
[40]	BALL P， PARKER-HOLDER J， PACCHIANO A， et al. Ready policy one： World building through active learning［C］∥ International Conference on Machine Learning. New York： PMLR， 2020.
[41]	HESTER T， QUINLAN M， STONE P. RTMBA： A real-time model-based reinforcement learning architecture for robot control［C］∥ 2012 IEEE International Conference on Robotics and Automation.Piscataway： IEEE Press， 2012.
[42]	WU J D， HUANG Z， LV C Y. Uncertainty-aware model-based reinforcement learning： Methodology and application in autonomous driving［J］. IEEE Transactions on Intelligent Vehicles， 2023， 8（1）： 194-203.
[43]	IMANBERDIYEV N， FU C H， KAYACAN E， et al. Autonomous navigation of UAV by using real-time model-based reinforcement learning［C］∥ 2016 14th International Conference on Control， Automation， Robotics and Vision （ICARCV）.Piscataway： IEEE Press， 2017.
[44]	WILLIAMS G， WAGENER N， GOLDFAIN B， et al. Information theoretic MPC for model-based reinforcement learning［C］∥ 2017 IEEE International Conference on Robotics and Automation （ICRA）. Piscataway： IEEE Press， 2017.
[45]	SERBAN I V， SANKAR C， PIEPER M， et al. The bottleneck simulator： A model-based deep reinforcement learning approach［J］. Journal of Artificial Intelligence Research， 2020， 69： 571-612.
[46]	陈旭东，陈琦琦，罗祎喆，等. 异构舰载机舰面保障作业动态并行调度［J］. 航空学报， 2025， 46（13）： 531329.
	CHEN X D， CHEN Q Q， LUO Y Z， et al. Dynamic parallel scheduling for heterogeneous carrier-based aircraft deck support operations［J］. Acta Aeronautica et Astronautica Sinica， 2025， 46（13）： 531329 （in Chinese）.

算法	场景1决策时间/s					场景2决策时间/s
算法	1架机	2架机	3架机	4架机	5架机	1架机	2架机	3架机	4架机	5架机
PPO	0.853	0.899	0.945	1.324	1.389	0.853	1.110	1.623	1.745	1.812
DQN	0.754	0.832	0.912	1.128	1.176	0.754	1.000	1.598	1.713	1.780
Double-DQN	0.780	0.849	0.983	1.236	1.289	0.780	1.053	1.617	1.735	1.795
Dueling-DQN	0.802	0.875	1.029	1.386	1.472	0.802	1.085	1.632	1.752	1.832
Dreamer	0.901	0.978	1.120	1.475	1.532	0.901	1.150	1.658	1.780	1.845
MB-MF	0.880	0.940	1.087	1.431	1.512	0.880	1.120	1.645	1.768	1.830
GA	92.210	145.730	192.570	217.380	298.240	92.210	150.730	315.870	356.480	383.250
PSO	80.380	132.450	160.710	174.260	259.970	80.380	135.120	270.840	315.680	372.430

场景	6架机出动时长/min	7架机出动时长/min	8架机出动时长/min
场景1	176	215	223
场景2	208	240	285

Scheduling algorithm for carrier-based aircraft support operations based on deck environment model

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 20

References 46

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Deqiang HAN, Wei XIONG, Ziling WANG. Low-attitude economy and information fusion in era of deep multi-modality [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(S1): 732853-732853.
[2]	Qinbo SUN, Zhaohui DANG. Spacecraft guardian strategy design via motion-intent recognition [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(S1): 733013-733013.
[3]	Long GAO, Chuanxiang ZHOU, Chaohui LIU, Wei ZHANG, Youbin LYU, Xuan LI, Tianyu LI, Xiaomei ZHENG. AADD: Aerial aircraft detection dataset [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(S1): 733255-733255.
[4]	Yuzhuo MA, Kan REN, Tao LI, Qian CHEN. Improving remote sensing image semantic segmentation based on distance loss [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(8): 332780-332780.
[5]	Yuheng LIU, Li YANG, Qilong HUANG. Optimizing air and missile defense strategies with explainable hierarchical reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(8): 332786-332786.
[6]	Hao ZHANG, Jianing LIU, Zhi XU, Yuanxin YANG. Trajectory prediction method of incoming missiles based on improved inverse reinforcement learning in aircraft active defense mode [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(8): 332753-332753.
[7]	Wei XIONG, Dong ZHANG, Shuheng YANG, Zhi REN, Wenyi LIU. Manned/unmanned aerial vehicle collaborative interpretable method for intelligent air combat [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(7): 332547-332547.
[8]	Yanlong HAN, An ZHANG, Wenhao BI, Qiucen FAN, Tianle HOU. Intelligent decision-making of airborne terminal infrared composite jamming based on DACTM-PPO [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(7): 332759-332759.
[9]	Sihua GAO, Bingyang ZHAO, Jianfu LI. UAV complete data collection trajectory planning algorithm based on time window constraints [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(6): 332451-332451.
[10]	Jian PENG, Guanglei ZHU, Qingshun WU, Yafei LI, Shuo HE, Yuanyuan JIN, Mingliang XU. Scheduling method for carrier-based aircraft support operations based on Monte Carlo tree search [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(6): 332444-332444.
[11]	Yunxiao LIAN, Ni LI, Feng XIE, Pan ZHOU, Changyin DONG. A multi-UAV cooperative air combat decision-making method based on spatial-temporal information fusion [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(6): 332633-332633.
[12]	Jun HUANG, Jing ZHANG, Shiqian WENG. Airborne electro-optical target recognition algorithms [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(6): 332601-332601.
[13]	Leyan LI, Rennong YANG, Anxin GUO, Qi SONG, Jialiang ZUO. Beyond-visual-range air combat threat prediction and dynamic evasion method based on all-domain fire field theory [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(4): 332205-332205.
[14]	Zicheng FENG, Wenlong ZHANG, Donghui LIU, Qifeng YU. Robust infrared target tracking algorithm for anti-UAV in complexbackgrounds [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(4): 332264-332264.
[15]	Lei ZHANG, Can TIAN, Fangqing WEN, Qinghe ZHANG, Han LIU. Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(3): 631880-631880.