关键词: 无人机, 航迹规划, 基站信道资源约束, 深度强化学习, 协同优化

Abstract:

To address the optimization problem of flight trajectories and communication strategies in multi-UAV collaborative reconnaissance missions， this study proposes a collaborative planning approach based on deep reinforcement learning， considering diverse costs such as flight distance， communication energy consumption， and channel capacity， along with multiple constraints including base station channel resource constraints， UAV performance constraints， and collision avoidance constraints. Firstly， a collaborative planning model for multi-UAV trajectories and communication strategies is established for randomly unknown reconnaissance environments. Secondly， an end-to-end deep reinforcement learning framework based on the multi-agent proximal policy optimization algorithm is introduced to jointly optimize coupled variables such as UAV trajectories， communication connection strategies， and communication transmit power， with flight distance， communication energy consumption， and channel capacity serving as multiple optimization objectives. To reduce the complexity of learning and solving multi-objective tasks， a trajectory planning sub-model incorporating virtual gravity from base stations is designed， which decreases the decision space. A trajectory planning sub-model that incorporates the virtual gravitational force of base stations is designed based on the artificial potential field method. Through the approach of automatically optimizing parameters via reinforcement learning， the size of the decision space is reduced， and the convergence speed of the model is accelerated. Finally， simulation experiments demonstrate the advantages of the proposed method in optimizing the overall mission cost index.

Key words: UVA, trajectory planning, base station channel resource constraint, deep reinforcement learning, collaborative optimization