Abstract: To address the insufficient exploration of subtask coupling relationships and limited dynamic adaptability in existing carrier-based aircraft support operation scheduling research, this study investigates a multi-stage scheduling problem for carrier-based aircraft support operations. Firstly, by modeling both support station allocation and aircraft servicing sequence determination as a multi-agent Markov decision process, we establish a mathematical characterization of the sequential coupling relationships between subtasks in support operation scheduling. Subsequently, an Independent Deep Q-network-based multi-agent collaborative decision-making framework is proposed, incorporating a distributed training-execution mechanism that specially includes a support station allocation module, an aircraft servicing sequence decision module, and a multi-agent collaborative scheduling module. Furthermore, a collaborative scheduling algorithm based on the multi-stage sequential decision-making mechanism is developed to solve the model. Finally, simulation results demonstrate that the proposed algorithm yields a 27.08% and 14.19% improvement in average reward, and a 56.44% and 45.43% improvement in reward standard deviation, over the Dueling DQN and N-step DQN methods, respectively, verifying the effectiveness of the multi-stage collaborative decision-making mechanism in addressing complex scheduling problems.

Key words: carrier-based aircraft, deep reinforcement learning, multi-stage, scheduling optimization, resource allocation