关键词: 近距空战, 智能决策, 强化学习, 种群博弈, SAC算法

Abstract:

With the development of artificial intelligence and Unmanned Aerial Vehicle （UAV） technologies， intelligent decision-making in close-range air combat has attracted extensive attention from all over the world. To solve the problems of overfitting and strategy cycles in using traditional reinforcement learning for intelligent decision-making in close-range air combat， a training paradigm of air combat decision model is proposed based on population game. By constructing a population composed of multiple UAV agents and assigning different reward weight coefficients to each agent， the diversified risk preference of UAV agents is realized. The problem of overfitting and strategy cycle can be avoided effectively by training agents of different risk preferences to fight against each other. During the training process， each UAV agent in the population adaptively optimizes the reward weight coefficient according to the results of the confrontation with different opponent strategies. In the numerical simulation experiment， Agent 5 and Agent 3 in population game training beat the intelligent decision model obtained by expert system adversarial training and self-play training with 88% and 85% success rate， respectively， which verifies the effectiveness of the algorithm. In addition， further experiments demonstrate the necessity of dynamic adjustment of weight coefficients in the training paradigm of population game， and verify the generality of the proposed training paradigm on heterogeneous models.

Key words: close-range air combat, intelligent decision-making, reinforcement learning, population game, SAC algorithm