Abstract: To address the coupling between dispatch decision optimization and tactical confrontation strategy optimization in multi-round UAV adversarial games, as well as the significant hierarchy between these two decision levels, this paper proposes a CFR-MAPPO hierarchical joint training method. At the strategic level, counterfactual regret minimization (CFR) is adopted to optimize discrete multi-round dispatch decisions with finite information sets. At the tactical level, multi-agent proximal policy optimization (MAPPO) is used to solve the trajectory planning problem in long-horizon continuous multi-UAV adversarial confrontation, thereby achieving coordinated integration between upper-level dispatch optimization and lower-level maneuver strategy learning. Simulation experiments are conducted in a nine-round continuous confrontation scenario involving 20 UAVs with identical performance on both sides, where each side is limited to no more than three UAVs per round. The results show that the proposed method achieves favorable performance in each single-round confrontation scenario, and the defending side successfully completes continuous defense with a probability of 73.5% despite being at a disadvantage. These results demonstrate the effectiveness of the proposed method and provide a complete framework and solution for the hierarchical optimization of UAV discrete decision-making and continuous adversarial confrontation.

Key words: UAV game confrontation, multi-round dispatch decision-making, counterfactual regret minimization (CFR), multi-agent proximal policy optimization (MAPPO), hierarchical collaborative training