基于深度强化学习的太阳能无人机航迹规划

doi:10.7527/S1000-6893.2024.31420

Abstract

Abstract:

High Altitude Long Endurance Solar-powered Unmanned Aerial Vehicles （HALE-SUAV） can significantly enhance the endurance performance through well-designed trajectory planning. Deep Reinforcement Learning （DRL） methods are ideal for this trajectory planning problem due to their real-time performance and adaptability. To address the HALE-SUAV trajectory planning problem based on DRL， this paper establishes the kinematics and dynamics models of the UAV， along with energy-related models， designs its energy management strategy， constructs the overall DRL framework for this trajectory planning problem， and ultimately conducts trajectory planning experiments under different solar radiation intensities using the trained model. The research results indicate that， based on the DRL method proposed in this paper， HALE-SUAVs can select reasonable control commands based on current solar radiation intensities to improve their endurance performance. The findings demonstrate the potential application value of DRL methods in HALE-SUAV trajectory planning problems.

Key words: deep reinforcement learning, HALE-SUAV, trajectory planning, endurance performance, energy management strategy

CLC Number:

V324.2

Zijie YU, Zheng ZHENG, Qingdong LI, Lin GUO, Suping REN, Jian GUO. Trajectory planning for solar-powered UAVs based on deep reinforcement learning[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(12): 331420.

Figures/Tables 16

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

HALE-SUAV related parameters

参数	符号	数值
总质量/kg	$m$	213
电池质量/kg	$m b e t t e r y$	141
最大飞行半径/km	$R m a x$	3
纵横比	$A R$	30
机翼面积/m²	$A$	60
翼展/m	$b$	42
电机效率	$η m o t o r$	0.95
最大飞行高度/m	$h m a x$	18 288
最小飞行高度/m	$h m i n$	24 288
最大攻角/ $°$	$α m a x$	10
最小攻角/ $°$	$α m i n$	0
最大倾斜角/ $°$	$ϕ m a x$	5
最小倾斜角/ $°$	$ϕ m i n$	-5
最大推力/N	$T p m a x$	500
最小推力/N	$T p m i n$	0
有效载荷质量/kg	$m p a y l o a d$	25
电池能量比/（（W·h）·kg^-1）	$U b a t t$	35
电池最大能量/（kW·h）	$E b a t t m a x$	59.5
最大放电功率/kW	$P b a t t e r y m a x$	11
最大充电功率/kW	$P b a t t e r y m i n$	-11
掠角/ $°$	$Λ$	17.5
有效载荷功率/W	$P p a y l o a d$	250
太阳能电池板面积/m²	$S$	60
螺旋桨半径/m	$r p r o p$	1.58

Table 1

Table 2

Values of fitting coefficients for aircraft lift surfaces

参数	数值
$a 1$	$3.774 21 × 10 - 1$
$a 2$	$1.243 16 × 10 - 1$
$a 3$	$7.646 15 × 10 - 7$
$a 4$	$- 5.682 28 × 10 - 3$
$a 5$	$- 6.445 53 × 10 - 13$
$a 6$	$- 2.650 58 × 10 - 8$

Table 2

Table 3

Values of fitting coefficients for aircraft drag surfaces

参数	数值
$b 1$	$6.448 15 × 10 - 2$
$b 2$	$- 1.878 41 × 10 - 7$
$b 3$	$1.793 26 × 10 - 13$
$b 4$	$- 1.113 85 × 10 - 2$
$b 5$	$3.750 46 × 10 - 8$
$b 6$	$- 3.105 91 × 10 - 14$
$b 7$	$1.097 53 × 10 - 3$
$b 8$	$- 2.367 96 × 10 - 9$
$b 9$	$1.584 61 × 10 - 15$

Table 3

Table 4

SMARTS model parameter setting information

参数	数值
仿真地点纬度/ $° N$	$35$
仿真地点经度/ $° W$	$106.6$
海拔高度/km	1.387
飞行高度/km	20
光谱范围/km	280~4 000
太阳辐照常数/km	1 366.1

Table 4

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

References 27

[1]	CESTINO E. Design of solar high altitude long endurance aircraft for multi payload & operations‍［J］. Aerospace Science and Technology， 2006， 10（6）： 541-550.
[2]	RAJENDRAN P， SMITH H. Implications of longitude and latitude on the size of solar-powered UAV［J］. Energy Conversion and Management， 2015， 98： 107-114.
[3]	高显忠，邓小龙，王玉杰，等. 临近空间太阳能飞机能量最优飞行航迹规划方法展望［J］. 航空学报， 2023， 44（8）： 027265.
	GAO X Z， DENG X L， WANG Y J， et al. General planning method for energy optimal flight path of solar-powered aircraft in near space‍［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（8）： 027265 （in Chinese）.
[4]	姚远，戴雨可，徐一鸣. 考虑不确定性的太阳能无人机总体参数设计［J］. 航空学报， 2024， 45（17）： 529856.
	YAO Y， DAI Y K， XU Y M. Overall parameter design of solar UAV considering uncertainty［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（17）： 529856 （in Chinese）.
[5]	李广佳，王红波，张凯，等. 临近空间太阳能无人机增升减阻技术综述［J］. 航空学报， 2024， 45（5）： 529644.
	LI G J， WANG H B， ZHANG K， et al. Lift enhancement and drag reduction technologies of solar powered unmanned aerial vehicles in near space： Review［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（5）： 529644 （in Chinese）.
[6]	SHIAU J K， MA D M， CHIU C W， et al. Optimal sizing and cruise speed determination for a solar-powered airplane［J］. Journal of Aircraft， 2010， 47（2）： 622-629.
[7]	LI X H， SUN K J， LI F. General optimal design of solar-powered unmanned aerial vehicle for priority considering propulsion system‍［J］. Chinese Journal of Aeronautics， 2020， 33（8）： 2176-2188.
[8]	KLESH A， KABAMBA P. Energy-optimal path planning for solar-powered aircraft in level flight［C］∥AIAA Guidance， Navigation and Control Conference and Exhibit. Reston： AIAA， 2007.
[9]	HUANG Y， CHEN J G， WANG H L， et al. A method of 3D path planning for solar-powered UAV with fixed target and solar tracking‍［J］. Aerospace Science and Technology， 2019， 92： 831-838.
[10]	SPANGELO S C， GILBERT E G. Power optimization of solar-powered aircraft with specified closed ground tracks［J］. Journal of Aircraft， 2012， 50（1）： 232-238.
[11]	GAO X Z， HOU Z X， GUO Z， et al. Energy management strategy for solar-powered high-altitude long-endurance aircraft‍［J］. Energy Conversion and Management， 2013， 70： 20-30.
[12]	LEE J S， YU K H. Optimal path planning of solar-powered UAV using gravitational potential energy‍［J］. IEEE Transactions on Aerospace and Electronic Systems， 2017， 53（3）： 1442-1451.
[13]	SUN M， JI X Z， SUN K W， et al. Flight strategy optimization for high-altitude solar-powered aircraft based on gravity energy reserving and mission altitude［J］. Applied Sciences， 2020， 10（7）： 2243.
[14]	WANG X Y， YANG Y P， WU D， et al. Mission-oriented 3D path planning for high-altitude long-endurance solar-powered UAVs with optimal energy management‍［J］. IEEE Access， 2020， 8： 227629-227641.
[15]	CAMCI E， KAYACAN E. End-to-end motion planning of quadrotors using deep reinforcement learning‍［DB/OL］. arXiv preprint： 1909.13599， 2019.
[16]	GANDHI D， PINTO L， GUPTA A. Learning to fly by crashing［C］∥2017 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. Piscataway： IEEE Press， 2017.
[17]	NG A Y， COATES A， DIEL M， et al. Autonomous inverted helicopter flight via reinforcement learning［M］∥ Experimental Robotics IX. Berlin： Springer Berlin Heidelberg， 2006： 363-372.
[18]	CLARKE S G， HWANG I. Deep reinforcement learning control for aerobatic maneuvering of agile fixed-wing aircraft［C］∥AIAA Scitech 2020 Forum. Reston： AIAA， 2020.
[19]	REDDY G， WONG-NG J， CELANI A， et al. Glider soaring via reinforcement learning in the field‍［J］. Nature， 2018， 562（7726）： 236-239.
[20]	BELLEMARE M G， CANDIDO S， CASTRO P S， et al. Autonomous navigation of stratospheric balloons using reinforcement learning‍［J］. Nature， 2020， 588（7836）： 77-82.
[21]	NI W J， BI Y， WU D， et al. Energy-optimal trajectory planning for solar-powered aircraft using soft actor-critic［J］. Chinese Journal of Aeronautics， 2022， 35（10）： 337-353.
[22]	MARTIN R A， GATES N S， NING A， et al. Dynamic optimization of high-altitude solar aircraft trajectories under station-keeping constraints［J］. Journal of Guidance， Control， and Dynamics， 2018， 42（3）： 538-552.
[23]	邵嘉琪，张晓辉，席涵宇，等. 太阳能无人机线性自抗扰多环路能源控制［J］. 航空学报， 2023， 44（10）： 327812.
	SHAO J Q， ZHANG X H， XI H Y， et al. Multi-loop energy control method of linear active disturbance rejection for solar-powered UAVs［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（10）： 327812 （in Chinese）.
[24]	SUTTON R S， BARTO A G. Reinforcement learning： An introduction‍［M］. 2nd ed. Cambridge： The MIT Press； 1998： 62-67.
[25]	HAARNOJA T， ZHOU A， ABBEEL P， et al. Soft actor-critic： Off-policy maximum entropy deep reinforcement learning with a stochastic actor［DB/OL］. arXiv preprint： 1801.01290，2018.
[26]	HAARNOJA T， ZHOU A， HARTIKAINEN K， et al. Soft actor-critic algorithms and applications［DB/OL］. arXiv preprint： 1812.05905， 2018.
[27]	SISSENWINE N， DUBIN M， WEXLER H. The U.S. standard atmosphere‍［J］. Journal of Geophysical Research， 1962， 67（9）： 3627-3630.

Trajectory planning for solar-powered UAVs based on deep reinforcement learning

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 16

References 27

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Kaifang WAN, Zhilin WU, Yunhui WU, Haozhi QIANG, Yibo WU, Bo LI. Cooperative location of multiple UAVs with deep reinforcement learning in GPS-denied environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331024-331024.
[2]	Lingfeng JIANG, Xinkai LI, Hai ZHANG, Hanwei LI, Hongli ZHANG. Mapless navigation of UAVs in dynamic environments based on an improved TD3 algorithm [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331035-331035.
[3]	Zhengyu SONG. Promoting continuous innovation in space transportation systems: Control technologies and challenges [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(6): 531446-531446.
[4]	Min YANG, Guanjun LIU, Ziyuan ZHOU. Control of lunar landers based on secure reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630553-630553.
[5]	Wenhui LING, Chunhui MU, Lingcong NIE, Xian DU, Ximing SUN. Improved DDPG-based multipoint pressure distribution control of variable geometry scramjet combustor at wide range velocities [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(12): 131092-131092.
[6]	Juntong WANG, Danwen BAO, Jiayi ZHOU, Jingxuan SHANG, Ziqian ZHANG. Low-altitude airspace planning: A review and prospect [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 530879-530879.
[7]	Junzhi LI, Teng LONG, Jingliang SUN, Hongyu MIAO, Zhenlin ZHOU. Differential flatness-based spatial-temporal hierarchical trajectory planning for fixed-wing UAVs in urban environments [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531369-531369.
[8]	Weicheng DI, Jinkui XU, Zixing WEI, Jinwu XIANG, Zhan TU. Aerial-ground heterogeneous cooperation based on multi-round task allocation method [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531348-531348.
[9]	Shuyi GAO, Defu LIN, Duo ZHENG, Cheng XU. Intelligent maneuvering penetration guidance strategies for aerial vehicles considering interceptor detection capability limitations [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 331304-331304.
[10]	Honglin ZHANG, Jianjun LUO, Weihua MA. Spacecraft game decision making for threat avoidance of space targets based on machine learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 329136-329136.
[11]	Yunpeng CAI, Dapeng ZHOU, Jiangchuan DING. Intelligent collaborative control of UAV swarms with collision avoidance safety constraints [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(5): 529683-529683.
[12]	Shengzhe SHAN, Weiwei ZHANG. Air combat intelligent decision-making method based on self-play and deep reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 328723-328723.
[13]	Yumei HU, Quan PAN, Bao DENG. A Fisher information based adaptive filtering algorithm for sensor trajectory planning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(20): 629825-629825.
[14]	Bing GAO, Zhejie ZHANG, Qijie ZOU, Zhiguo LIU, Xiling ZHAO. Multi-agent communication cooperation based on deep reinforcement learning and information theory [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(18): 329862-329862.
[15]	Zuolong LI, Jihong ZHU, Minchi KUANG, Jie ZHANG, Jie REN. Hierarchical decision algorithm for air combat with hybrid action based on deep reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(17): 530053-530053.