Abstract:

The goal of opponent modeling is to model the opponent’s strategy to maximize the payoff of the main intelligent agent. Most previous work has failed to effectively handle the situations where opponent information is limited. To address this problem， we propose an approach for opponent modeling with incomplete information （OMII）， which is capable of extracting cross-epoch opponent strategy representations by solely using self-observations in case of limited opponent information. OMII introduces a novel policy-based data augmentation method that， through contrastive learning， offline learns opponent strategy representations and employs them as additional input to train a general responsive policy. During online testing， OMII extracts opponent strategy representations from recent historical trajectory data and dynamically responds to opponent strategies in conjunction with the general policy. Moreover， OMII ensures a lower bound on the expected payoff by balancing conservatism and exploitation. Experimental results demonstrate that even in scenarios with limited opponent information， OMII can accurately extract opponent strategy representations， and possesses certain generalization capabilities for unknown strategies， outperforming existing opponent modeling algorithms in performance.

Key words: decision making intelligence, reinforcement learning, opponent modeling, constrastive learning, multiagent system