ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Spacecraft game decision making for threat avoidance of space targets based on machine learning
Received date: 2023-06-06
Revised date: 2023-08-22
Accepted date: 2023-11-02
Online published: 2023-11-16
Supported by
National Natural Science Foundation of China(12072269);Foundation of Science and Technology on Aerospace Flight Dynamics Laboratory(6142210210302)
An intelligent decision-making framework and a deep reinforcement learning-based autonomous decision-making method are proposed for the spacecraft decision-making in avoiding the threat of space targets. Taking into account the maneuvering characteristics of space targets and the gameplay of threat avoidance， an intelligent game decision-making framework for spacecraft threat avoidance is proposed based on the Observation-Orientation-Decision-Action （OODA） loop decision-making idea and machine learning techniques. Based on this framework and inference on the motion intentions of space targets， a deep reinforcement learning-based spacecraft maneuver decision-making algorithm and training environment are designed to enable spacecraft decision-making control with game response capability， which realizes the avoidance response to the typical motion intentions of space targets. Furthermore， the generalization of spacecraft autonomous maneuvering decision-making algorithm and its adaptability to possible uncertain maneuvers of space targets are improved by using the self-play learning technique. Finally， the effectiveness of our proposed method is verified through simulations.
Honglin ZHANG , Jianjun LUO , Weihua MA . Spacecraft game decision making for threat avoidance of space targets based on machine learning[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2024 , 45(8) : 329136 -329136 . DOI: 10.7527/S1000-6893.2023.29136
| 1 | 袁利， 姜甜甜. 航天器威胁规避智能自主控制技术研究综述［J］. 自动化学报， 2023， 49（2）： 229-245. |
| YUAN L， JIANG T T. Review on intelligent autonomous control for spacecraft confronting orbital threats［J］. Acta Automatica Sinica， 2023， 49（2）： 229-245 （in Chinese）. | |
| 2 | 袁利. 面向不确定环境的航天器智能自主控制技术［J］. 宇航学报， 2021， 42（7）： 839-849. |
| YUAN L. Spacecraft intelligent autonomous control technology toward uncertain environment［J］. Journal of Astronautics， 2021， 42（7）： 839-849 （in Chinese）. | |
| 3 | 王杰， 丁达理， 董康生， 等. UCAV自主空战战术机动动作建模与轨迹生成［J］. 火力与指挥控制， 2018， 43（12）： 42-49. |
| WANG J， DING D L， DONG K S， et al. UCAV autonomous air combat tactical maneuvering modeling and trajectory generation［J］. Fire Control & Command Control， 2018， 43（12）： 42-49 （in Chinese）. | |
| 4 | 于大腾， 王华， 孙福煜. 考虑潜在威胁区的航天器最优规避机动策略［J］. 航空学报， 2017， 38（1）： 320202. |
| YU D T， WANG H， SUN F Y. Optimal evasive maneuver strategy with potential threatening area being considered［J］. Acta Aeronautica et Astronautica Sinica， 2017， 38（1）： 320202 （in Chinese）. | |
| 5 | BOMBARDELLI C. Analytical formulation of impulsive collision avoidance dynamics［J］. Celestial Mechanics and Dynamical Astronomy， 2014， 118（2）： 99-114. |
| 6 | GONZALO J L， COLOMBO C， DI LIZIA P. Analytical framework for space debris collision avoidance maneuver design［J］. Journal of Guidance， Control， and Dynamics， 2020， 44（3）： 469-487. |
| 7 | BATHER J A， ISAACS R. Differential games： a mathematical theory with applications to warfare and pursuit， control and optimization［J］. Journal of the Royal Statistical Society Series A （General）， 1966， 129（3）： 474. |
| 8 | PRINCE E R， HESS J A， COBB R G， et al. Elliptical orbit proximity operations differential games［J］. Journal of Guidance， Control， and Dynamics， 2019， 42（7）： 1458-1472. |
| 9 | LIANG L， DENG F， PENG Z H， et al. A differential game for cooperative target defense［J］. Automatica， 2019， 102： 58-71. |
| 10 | SUN J L， LIU C S， YE Q. Robust differential game guidance laws design for uncertain interceptor-target engagement via adaptive dynamic programming［J］. International Journal of Control， 2017， 90（5）： 990-1004. |
| 11 | WATANABE T， JOHNSON E N. Trajectory generation using deep neural network： AIAA-2018-1893［R］. Reston： AIAA， 2018. |
| 12 | IZZO D， TAILOR D， VASILEIOU T. On the stability analysis of deep neural network representations of an optimal state-feedback［DB/OL］. arXiv preprint： 1812.02532， 2018. |
| 13 | SáNCHEZ-SáNCHEZ C， IZZO D. Real-time optimal control via deep neural networks： Study on landing problems［J］. Journal of Guidance， Control， and Dynamics， 2018， 41（5）： 1122-1135. |
| 14 | OESTREICH C E， LINARES R， GONDHALEKAR R. Autonomous six-degree-of-freedom spacecraft docking with rotating targets via reinforcement learning［J］. Journal of Aerospace Information Systems， 2021， 18（7）： 417-428. |
| 15 | 刘冰雁， 叶雄兵， 高勇， 等. 基于分支深度强化学习的非合作目标追逃博弈策略求解［J］. 航空学报， 2020， 41（10）： 324040. |
| LIU B Y， YE X B， GAO Y， et al. Strategy solution of non-cooperative target pursuit-evasion game based on branching deep reinforcement learning［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（10）： 324040 （in Chinese）. | |
| 16 | ZHANG J R， ZHANG K P， ZHANG Y， et al. Near-optimal interception strategy for orbital pursuit-evasion using deep reinforcement learning［J］. Acta Astronautica， 2022， 198： 9-25. |
| 17 | 赵毓， 郭继峰， 颜鹏， 等. 稀疏奖励下多航天器规避决策自学习仿真［J］. 系统仿真学报， 2021， 33（8）： 1766-1774. |
| ZHAO Y， GUO J F， YAN P， et al. Self-learning-based multiple spacecraft evasion decision making simulation under sparse reward condition［J］. Journal of System Simulation， 2021， 33（8）： 1766-1774 （in Chinese）. | |
| 18 | ZHANG H L， LUO J J， GAO Y， et al. An intention inference method for the space non-cooperative target based on BiGRU-Self Attention［J］. Advances in Space Research， 2023， 72（5）： 1815-1828. |
| 19 | 黎飞， 雷拥军， 冯佳佳. 一种GEO卫星太阳光遮挡轨迹设计与控制方法［J］. 宇航学报， 2022， 43（2）： 198-205. |
| LI F， LEI Y J， FENG J J. A design and control method of Sun occlusion trajectory for GEO satellite［J］. Journal of Astronautics， 2022， 43（2）： 198-205 （in Chinese）. | |
| 20 | SILVER D， SCHRITTWIESER J， SIMONYAN K， et al. Mastering the game of Go without human knowledge［J］. Nature， 2017， 550： 354-359. |
/
| 〈 |
|
〉 |
CJA