ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Trajectory planning of solar powered unmanned aerial vehicles based on multi-objective reinforcement learning
Received date: 2025-09-23
Revised date: 2025-11-13
Accepted date: 2025-12-09
Online published: 2025-12-23
There is a significant coupling between the influencing factors of high-altitude long-endurance solar-powered UAVs in harvesting solar energy and gradient wind energy， and optimizing the harvesting efficiency of these two types of energy simultaneously often leads to conflicts. To address this issue， this study proposes a trajectory planning method based on multi-objective reinforcement learning. This method adopts the multi-objective Soft Actor-Critic （SAC） algorithm based on the multi-objective Markov decision process， combines the UAV's energy harvesting power and energy consumption power into a reward vector， and adds randomly generated weights in each update step. The converged trained policy network can output thrust， attack angle， and bank angle commands based on flight information and a given weight vector， enabling the generation of a set of energy-optimal trajectory solutions within the weight space. Simulation results show that compared with the minimum energy consumption strategy and the strategy based on the conventional single-objective SAC algorithm， this method consistently achieves better energy optimization efficiency and can adaptively respond to weight changes of energy objectives. Compared with the offline optimized trajectory solution set based on the Non-dominated Sorting Genetic Algorithm Ⅱ， this method achieves a hypervolume of the trajectory solution set reaching 90.07% of the former while maintaining excellent real-time performance. In addition， this method also demonstrates a certain degree of generalization ability and can adapt to new untrained wind fields.
Tichao XU , Wenyue MENG , Jian ZHANG . Trajectory planning of solar powered unmanned aerial vehicles based on multi-objective reinforcement learning[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2026 , 47(12) : 332817 -332817 . DOI: 10.7527/S1000-6893.2025.32817
| [1] | KLESH A， KABAMBA P. Energy-optimal path planning for solar-powered aircraft in level flight［C］∥AIAA Guidance， Navigation and Control Conference and Exhibit. Reston： AIAA， 2007. |
| [2] | EDWARDS D J， KAHN A D， KELLY M， et al. Maximizing net power in circular turns for solar and autonomous soaring aircraft［J］. Journal of Aircraft， 2016， 53（5）： 1237-1247. |
| [3] | AILON A. A path planning approach for unmanned solar-powered aerial vehicles［J］. Renewable Energy Power Quality Journal， 2023， 21： 109-114. |
| [4] | SPANGELO S C， GILBERT E G. Power optimization of solar-powered aircraft with specified closed ground tracks［J］. Journal of Aircraft， 2013， 50（1）： 232-238. |
| [5] | HUANG Y， CHEN J G， WANG H L， et al. A method of 3D path planning for solar-powered UAV with fixed target and solar tracking［J］. Aerospace Science and Technology， 2019， 92： 831-838. |
| [6] | MARTIN R A， GATES N S， NING A， et al. Dynamic optimization of high-altitude solar aircraft trajectories under station-keeping constraints［J］. Journal of Guidance， Control， and Dynamics， 2019， 42（3）： 538-552. |
| [7] | BOLANDHEMMAT H， THOMSEN B， MARRIOTT J. Energy-optimized trajectory planning for high altitude long endurance （HALE） aircraft［C］∥2019 18th European Control Conference （ECC）. Piscataway： IEEE Press， 2019： 1486-1493. |
| [8] | SACHS G， LENZ J， HOLZAPFEL F. Unlimited endurance performance of solar UAVs with minimal or zero electrical energy storage［C］∥AIAA Guidance， Navigation， and Control Conference. Reston： AIAA， 2009. |
| [9] | GAO X Z， HOU Z X， GUO Z， et al. Joint optimization of battery mass and flight trajectory for high-altitude solar-powered aircraft［J］. Proceedings of the Institution of Mechanical Engineers， Part G： Journal of Aerospace Engineering， 2014， 228（13）： 2439-2451. |
| [10] | 王少奇， 马东立， 杨穆清， 等. 高空太阳能无人机三维航迹优化［J］. 北京航空航天大学学报， 2019， 45（5）： 936-943. |
| WANG S Q， MA D L， YANG M Q， et al. Three-dimensional optimal path planning for high-altitude solar-powered UAV［J］. Journal of Beijing University of Aeronautics and Astronautics， 2019， 45（5）： 936-943 （in Chinese）. | |
| [11] | MARRIOTT J， TEZEL B， LIU Z， et al. Trajectory optimization of solar-powered high-altitude long endurance aircraft［C］∥2020 6th International Conference on Control， Automation and Robotics （ICCAR）. Piscataway： IEEE Press， 2020： 473-481. |
| [12] | NI W J， BI Y， WU D， et al. Energy-optimal trajectory planning for solar-powered aircraft using soft actor-critic［J］. Chinese Journal of Aeronautics， 2022， 35（10）： 337-353. |
| [13] | RICHARDSON P L. Upwind dynamic soaring of albatrosses and UAVs［J］. Progress in Oceanography， 2015， 130： 146-156. |
| [14] | SACHS G， LESCH K， KNOLL A. Optimal control for maximum energy extraction from wind shear［C］∥Guidance， Navigation and Control Conference. Reston：AIAA， 1989. |
| [15] | SACHS G. Minimum shear wind strength required for dynamic soaring of albatrosses［J］. Ibis， 2005， 147（1）： 1-10. |
| [16] | LIU D N， HOU Z X， GAO X Z. Flight modeling and simulation for dynamic soaring with small unmanned air vehicles［J］. Proceedings of the Institution of Mechanical Engineers， Part G： Journal of Aerospace Engineering， 2017， 231（4）： 589-605. |
| [17] | NIE X Q， ZWENIG A， PIPREK P， et al. Dynamic soaring trajectory optimization considering the path following performance［J］. IEEE Transactions on Aerospace and Electronic Systems， 2025， 61（4）： 9184-9201. |
| [18] | ZWENIG A， BEYER Y， HONG H C， et al. Trajectory optimization of dynamic soaring considering rigid body and actuator dynamics using warmstarting［C］∥AIAA SCITECH 2025 Forum. Reston： AIAA， 2025. |
| [19] | ZHONG G X， XI H Z， ZHENG G， et al. The influence of wind shear to the performance of high-altitude solar-powered aircraft［J］. Proceedings of the Institution of Mechanical Engineers， Part G： Journal of Aerospace Engineering， 2014， 228（9）： 1562-1573. |
| [20] | ZOU W Y， LI N， AN F C， et al. A novel trajectories optimizing method for dynamic soaring based on deep reinforcement learning［J］. Defence Technology， 2025， 46： 99-108. |
| [21] | PARK S， FANJOY A， GOLUBEV V V. Application of reinforcement learning for autonomous dynamic soaring［C］∥AIAA SCITECH 2025 Forum. Reston： AIAA， 2025. |
| [22] | BOWER G， FLANZER T， KROO I. Conceptual design of a small UAV for continuous flight over the ocean［C］∥11th AIAA Aviation Technology， Integration， and Operations （ATIO） Conference. Reston： AIAA， 2011. |
| [23] | 刘多能. 固定翼无人机动态滑翔机理与航迹优化研究［D］. 长沙： 国防科学技术大学， 2016. |
| LIU D N. Research on mechanism and trajectory optimization for dynamic soaring with fixed-wing unmanned aerial vehicles［D］. Changsha： National University of Defense Technology， 2016 （in Chinese）. | |
| [24] | 刘思奇， 白俊强. 结合动态滑翔技术的小型太阳能无人机飞行能量变化分析［J］. 西北工业大学学报， 2020， 38（1）： 48-57. |
| LIU S Q， BAI J Q. Analysis of flight energy variation of small solar UAVs using dynamic soaring technology［J］. Journal of Northwestern Polytechnical University， 2020， 38（1）： 48-57 （in Chinese）. | |
| [25] | 张云飞， 王宏伦， 张梦华， 等. 基于强化学习的多能源动态滑翔航迹优化方法［J］. 西北工业大学学报， 2025， 43（1）： 128-139. |
| ZHANG Y F， WANG H L， ZHANG M H， et al. Multi energy dynamic soaring trajectory optimization method based on reinforcement learning［J］. Journal of Northwestern Polytechnical University， 2025， 43（1）： 128-139 （in Chinese）. | |
| [26] | YAN Z J， TABASSUM H. Generalized multi-objective reinforcement learning with envelope updates in URLLC-enabled vehicular networks［J］. IEEE Transactions on Vehicular Technology， 2025， 74（11）： 17666-17682. |
| [27] | LI F Y， MA R C， DU J R， et al. Decomposition-based multi-objective reinforcement learning for dynamic disassembly job shop scheduling with urgency guidance［J］. Swarm and Evolutionary Computation， 2025， 97： 102040. |
| [28] | KEMPER N， HEIDER M， PIETRUSCHKA D， et al. A comparative study of multi-objective and neuroevolutionary-based reinforcement learning algorithms for optimizing electric vehicle charging and load management［J］. Applied Energy， 2025， 391： 125890. |
| [29] | SUN G， XIAO J， LI J H， et al. Aerial reliable collaborative communications for terrestrial mobile users via evolutionary multi-objective deep reinforcement learning［J］. IEEE Transactions on Mobile Computing， 2025， 24（7）： 5731-5748. |
| [30] | CLARK C A， ALBARADO K M， WILSON J P， et al. Explainability for unmanned aerial vehicle control via multi-objective reinforcement learning［C］∥2025 IEEE Aerospace Conference. Piscataway： IEEE Press， 2025. |
| [31] | XI Z Y， WU D， NI W J， et al. Energy-optimized trajectory planning for solar-powered aircraft in a wind field using reinforcement learning［J］. IEEE Access， 2022， 10： 87715-87732. |
| [32] | ETKIN B. Dynamics of atmospheric flight： Chelmsford［M］. North Chelmsford： Courier Corporation， 2012：149-150. |
| [33] | KEIDEL B. Design and simulation of high-altitude long-endurance solar-powered drones solardrohnen［D］. Munich： Technical University of Munich， 2000 （in Genman）. |
| [34] | LU H Y， HERMAN D， YU Y L. Multi-objective reinforcement learning： Convexity， stationarity and Pareto optimality［C］∥International Conference on Learning Representations， 2023. |
| [35] | 刘思奇， 白俊强. 基于六自由度模型的高空动态滑翔探究［J］. 西北工业大学学报， 2021， 39（4）： 703-711. |
| LIU S Q， BAI J Q. Exploration of high-altitude dynamic soaring based on six-degree-of-freedom model［J］. Journal of Northwestern Polytechnical University， 2021， 39（4）： 703-711 （in Chinese）. | |
| [36] | PARK S， DEYST J， HOW J. A new nonlinear guidance logic for trajectory tracking［C］∥AIAA Guidance， Navigation， and Control Conference and Exhibit. Reston： AIAA， 2004. |
| [37] | DEB K， PRATAP A， AGARWAL S， et al. A fast and elitist multiobjective genetic algorithm： NSGA-Ⅱ［J］. IEEE Transactions on Evolutionary Computation， 2002， 6（2）： 182-197. |
/
| 〈 |
|
〉 |
CJA