Abstract
With the rapid advancement of technology, aerial interaction patterns have become increasingly complex, making intelligent air combat a prominent and cutting-edge research area in multi-agent systems. In this context, the dynamic and uncertain nature of large-scale air combat scenarios poses significant challenges, including scalability issues, computational complexity, and decision-making difficulties in multi-agent collaborative decision-making. To address these challenges, we propose a novel multi-aircraft autonomous decision-making approach based on graphs and multi-agent reinforcement learning (MADRL) under zero-order optimization, implemented through the GraphZero-PPO algorithm. This method innovatively integrates GraphSAGE and zero-order optimization into the MADRL framework. By leveraging the graph structure to adapt to the high dynamics and high-dimensional characteristics of multi-agent systems, the proposed approach enables rapid decision-making for missile launches through an efficient sampling strategy while employing zero-order optimization to explore global optima effectively. Finally, we present simulation experiments conducted in both 1v1 and 8v8 air combat scenarios, along with comparative results. The findings demonstrate that the proposed method can effectively adapt to large-scale air combat environments while achieving high win rates and rapid decision-making performance.