导航

ACTA AERONAUTICAET ASTRONAUTICA SINICA ›› 2021, Vol. 42 ›› Issue (4): 524009-524009.doi: 10.7527/S1000-6893.2020.24009

• Electronics and Electrical Engineering and Control • Previous Articles     Next Articles

Coordination control method for fixed-wing UAV formation through deep reinforcement learning

XIANG Xiaojia, YAN Chao, WANG Chang, YIN Dong   

  1. College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China
  • Received:2020-03-24 Revised:2020-05-18 Published:2020-07-06
  • Supported by:
    National Natural Science Foundation of China (61906203); The Foundation of National Key Laboratory of Science and Technology on UAV, Northwestern Polytechnical University (614230110080817)

Abstract: Due to the complexity of kinematics and environmental dynamics, controlling a squad of fixed-wing Unmanned Aerial Vehicles (UAVs) remains a challenging problem. Considering the uncertainty of complex and dynamic environments, this paper solves the coordination control problem of UAV formation based on the model-free deep reinforcement learning algorithm. A new action selection strategy, ε-imitation strategy, is proposed by combining the ε-greedy strategy and the imitation strategy to balance the exploration and the exploitation. Based on this strategy, the double Q-learning technique, and the dueling architecture, the ID3QN (Imitative Dueling Double Deep Q-Network) algorithm is developed to boost learning efficiency. The results of the Hardware-In-Loop experiments conducted in a high-fidelity semi-physical simulation system demonstrate the adaptability and practicality of the proposed ID3QN coordinated control algorithm.

Key words: fixed-wing UAVs, UAV formation, coordination control, deep reinforcement learning, neural networks

CLC Number: