面向移动边缘网络的多目标进化深度确定性策略梯度算法

doi:10.7527/S1000-6893.2025.31880

Abstract

Abstract:

The Mobile Edge Computing （MEC） network assisted by Unmanned Aerial Vehicles （UAV）demonstrates great potential in emergency response， real-time monitoring， and other fields. However， the efficient operation of MEC network encounters challenges stemming from multiple optimization objectives， such as high energy consumption and high latency. Therefore， a Multi-Objective Evolution with Deep Deterministic Policy Gradient （MOE-DDPG） algorithm for UAV-assisted MEC network optimization is introduced. Firstly， an integrated multi-objective optimization model is established to ensure comprehensive performance of the MEC network by minimizing latency and energy consumption while maximizing the number of completed UAV tasks. Secondly， a bidirectional selection strategy for weight vector and individual matching is proposed to address the difficulty of balancing various objectives in traditional Deep Deterministic Policy Gradient （DDPG） algorithms when dealing with multi-objective optimization problems， thereby significantly enhancing population diversity. Finally， by organically fusing the Multi-Objective Evolution （MOE） algorithm and DDPG algorithm， a novel MOE-DDPG algorithm framework is proposed， which can optimize the overall performance of the MEC network in real time. The experimental results show that the MOE-DDPG algorithm not only significantly improves the distribution and convergence of the Pareto solution set but also effectively reduces energy consumption， latency， and increases the number of completed tasks.

Key words: deep reinforcement learning, Mobile Edge Computing (MEC), unmanned aerial vehicle, Multi-Objective Evolution (MOE), bidirectional selection

CLC Number:

V249

Lei ZHANG, Can TIAN, Fangqing WEN, Qinghe ZHANG, Han LIU. Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks[J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(3): 631880.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Table 1

Experimental parameters

参数	符号	数值
UAV任务队列容量	$T u$	10
最大方位角/（°）	$θ m a x$	π/4
旋翼叶片剖面功率/W	$P s$	79.86
旋翼叶片尖端速度/（m·s^-1）	$U t i p$	120
诱导功率/W	$P d$	88.63
平均诱导速度/（m·s^-1）	$v 0$	4.03
机身阻力系数	$C D$	0.6
空气密度/（kg·m^-3）	$ρ$	1.225
旋翼固有比	$r c$	0.05
旋翼面积/m²	$μ$	0.503
信道带宽/MHz	$B$	10
UAV发射功率/W	$P T$	1
UAV与BS之间的信道噪声功率/（10^-6 W）	$σ u b$	1
IoTTs发射功率/W	$P u p$	1
UAV计算能力/GHz	$f U$	1
有效电容系数/10^-26	$κ$	1
折扣因子	$γ$	0.995
软更新比例	$ε$	0.005
正常数	$φ$	2 000
准备阶段最大更新次数	$U s$	60
进化阶段最大更新次数	$U e$	10
最大进化代数	$G m a x$	100

Table 1

Fig.4

Fig.5

Table 2

Fig.6

Fig.7

References 36

[1]	CAO B Q， YE H F， LIU J X， et al. SMART： Cost-aware service migration path selection based on deep reinforcement learning［J］. IEEE Transactions on Intelligent Transportation Systems， 2024， 25（9）： 12421-12436.
[2]	CHENG S Y， REN T， ZHANG H， et al. A Stackelberg-game-based framework for edge pricing and resource allocation in mobile edge computing［J］. IEEE Internet of Things Journal， 2024， 11（11）： 20514-20530.
[3]	李伟，郭艳，李宁，等. 智能反射面辅助无人机移动边缘计算任务数据最大化方法［J］. 航空学报， 2023， 44（19）： 328486.
	LI W， GUO Y， LI N， et al. Intelligent reflector surface assisted UAV mobile edge computing task data maximization method［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（19）： 328486 （in Chinese）.
[4]	SHAH Z， JAVED U， NAEEM M， et al. Mobile edge computing （MEC）-enabled UAV placement and computation efficiency maximization in disaster scenario［J］. IEEE Transactions on Vehicular Technology， 2023， 72（10）： 13406-13416.
[5]	ZHAI J H， BI J， YUAN H T， et al. Cost-minimized microservice migration with autoencoder-assisted evolution in hybrid cloud and edge computing systems［J］. IEEE Internet of Things Journal， 2024， 11（24）： 40951-40967.
[6]	李伟，郭艳，何明，等. 满意度驱动下无人机移动边缘计算服务缓存和资源分配方法［J］. 航空学报， 2024， 45（19）： 330017.
	LI W， GUO Y， HE M， et al. Satisfaction-driven services caching and resource allocation for UAV mobile edge computing［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（19）： 330017 （in Chinese）.
[7]	HUI M， CHEN J， YANG L， et al. UAV-assisted mobile edge computing： Optimal design of UAV altitude and task offloading［J］. IEEE Transactions on Wireless Communications， 2024， 23（10）： 13633-13647.
[8]	屈毓锛，秦蓁，马靖豪，等. 面向空地协同移动边缘计算的服务布置策略［J］. 计算机学报， 2022， 45（4）： 781-797.
	QU Y B， QIN Z， MA J H， et al. Service provisioning for air-ground collaborative mobile edge computing［J］. Chinese Journal of Computers， 2022， 45（4）： 781-797 （in Chinese）.
[9]	CAO L M， HUO T， LI S B， et al. Cost optimization in edge computing： A survey［J］. Artificial Intelligence Review， 2024， 57（11）： 312.
[10]	CHAI Z Y， LIU X， LI Y L. A computation offloading algorithm based on multi-objective evolutionary optimization in mobile edge computing［J］. Engineering Applications of Artificial Intelligence， 2023， 121： 105966.
[11]	CUI Y Y， ZHANG D G， ZHANG T， et al. A novel offloading scheduling method for mobile application in mobile edge computing［J］. Wireless Networks， 2022， 28（6）： 2345-2363.
[12]	FU S， ZHOU F H， HU R Q Y. Resource allocation in a relay-aided mobile edge computing system［J］. IEEE Internet of Things Journal， 2022， 9（23）： 23659-23669.
[13]	EJAZ M， GUI J S， ASIM M， et al. RL-planner： Reinforcement learning-enabled efficient path planning in multi-UAV MEC systems［J］. IEEE Transactions on Network and Service Management， 2024， 21（3）： 3317-3329.
[14]	CHEN C， GONG S M， ZHANG W J， et al. DRL-based contract incentive for wireless-powered and UAV-assisted backscattering MEC system［J］. IEEE Transactions on Cloud Computing， 2024， 12（1）： 264-276.
[15]	LI B， LIU Y F， TAN L， et al. Digital twin assisted task offloading for aerial edge computing and networks［J］. IEEE Transactions on Vehicular Technology， 2022， 71（10）： 10863-10877.
[16]	WANG L Y， ZHANG G L. Joint service caching， resource allocation and computation offloading in three-tier cooperative mobile edge computing system［J］. IEEE Transactions on Network Science and Engineering， 2023， 10（6）： 3343-3353.
[17]	LI M， YU F R， SI P B， et al. UAV-assisted data transmission in blockchain-enabled M2M communications with mobile edge computing［J］. IEEE Network， 2020， 34（6）： 242-249.
[18]	ZHANG S X， CAO R Y. Multi-objective optimization for UAV-enabled wireless powered IoT networks： An LSTM-based deep reinforcement learning approach［J］. IEEE Communications Letters， 2022， 26（12）： 3019-3023.
[19]	LI L P， GUAN W Q， ZHAO C， et al. Trajectory planning， phase shift design， and IoT devices association in flying-RIS-assisted mobile edge computing［J］. IEEE Internet of Things Journal， 2024， 11（1）： 147-157.
[20]	ZHANG S， JIN H L， GUO P K. IRS-assisted energy efficient communication for UAV mobile edge computing［J］. Computer Networks， 2024， 246： 110387.
[21]	AL-HILO A， SAMIR M， ELHATTAB M， et al. RIS-assisted UAV for timely data collection in IoT networks［J］. IEEE Systems Journal， 2023， 17（1）： 431-442.
[22]	SHI J L， LI C Y， GUAN Y C， et al. Multi-UAV-assisted computation offloading in DT-based networks： A distributed deep reinforcement learning approach［J］. Computer Communications， 2023， 210： 217-228.
[23]	ZHAO X， ZHAO T H， WANG F Y， et al. SAC-based UAV mobile edge computing for energy minimization and secure data transmission［J］. Ad Hoc Networks， 2024， 157： 103435.
[24]	LI B， XIE W C， YE Y H， et al. FlexEdge： Digital twin-enabled task offloading for UAV-aided vehicular edge computing［J］. IEEE Transactions on Vehicular Technology， 2023， 72（8）： 11086-11091.
[25]	JIANG W W， AI B， LI M S， et al. Aerial-IRSs-assisted energy-efficient task offloading and computing［J］. IEEE Internet of Things Journal， 2024， 11（11）： 20178-20193.
[26]	YU Y， TANG J， HUANG J Y， et al. Multi-objective optimization for UAV-assisted wireless powered IoT networks based on extended DDPG algorithm［J］. IEEE Transactions on Communications， 2021， 69（9）： 6361-6374.
[27]	ZHOU C H， WU W， HE H L， et al. Deep reinforcement learning for delay-oriented IoT task scheduling in SAGIN［J］. IEEE Transactions on Wireless Communications， 2021， 20（2）： 911-925.
[28]	季薇，杨许鑫，李飞，等. 无人机辅助MEC系统中基于最优SIC顺序的能耗优化方案［J］. 通信学报， 2024， 45（2）： 18-30.
	JI W， YANG X X， LI F， et al. Energy consumption optimization scheme in UAV-assisted MEC system based on optimal SIC order［J］. Journal on Communications， 2024， 45（2）： 18-30 （in Chinese）.
[29]	YI X J， YU H Y， XU T. Solving multi-objective weapon-target assignment considering reliability by improved MOEA/D-AM2M［J］. Neurocomputing， 2024， 563： 126906.
[30]	WANG P F， YANG H， HAN G J， et al. Decentralized navigation with heterogeneous federated reinforcement learning for UAV-enabled mobile edge computing［J］. IEEE Transactions on Mobile Computing， 2024， 23（12）： 13621-13638.
[31]	HAO H， XU C Q， ZHANG W， et al. Joint task offloading， resource allocation， and trajectory design for multi-UAV cooperative edge computing with task priority［J］. IEEE Transactions on Mobile Computing， 2024， 23（9）： 8649-8663.
[32]	PERVEZ F， SULTANA A， YANG C G， et al. Energy and latency efficient joint communication and computation optimization in a multi-UAV-assisted MEC network［J］. IEEE Transactions on Wireless Communications， 2024， 23（3）： 1728-1741.
[33]	SONG F H， DENG M S， XING H L， et al. Energy-efficient trajectory optimization with wireless charging in UAV-assisted MEC based on multi-objective reinforcement learning［J］. IEEE Transactions on Mobile Computing， 2024， 23（12）： 10867-10884.
[34]	JIN J L， XU Y J. Optimal policy characterization enhanced proximal policy optimization for multitask scheduling in cloud computing［J］. IEEE Internet of Things Journal， 2022， 9（9）： 6418-6433.
[35]	SHEN S G， HAO X B， GAO Z J， et al. SAC-PP： Jointly optimizing privacy protection and computation offloading for mobile edge computing［J］. IEEE Transactions on Network and Service Management， 2024， 21（6）： 6190-6203.
[36]	TAN L， GUO S T， ZHOU P Z， et al. HAT： Task offloading and resource allocation in RIS-assisted collaborative edge computing［J］. IEEE Transactions on Network Science and Engineering， 2024， 11（5）： 4665-4678.

算法	ACOI	ATD	AEC	ANTC
MOE-DDPG	592.242 1（+8.5%）	102.614 2（+12.6%）	324.237 1（+10.3%）	348.666 7（+22.5%）
MOE-PPO	646.971 1	117.368 7	351.751 8（+2.7%）	284.733 3
MOE-SAC	617.546 4（+4.5%）	112.478 0（+4.2%）	351.142 9（+2.8%）	328.666 7（+15.4%）
MOE-TD3	608.619 5（+5.9%）	91.111 9（+22.4%）	361.424 3	327.533 3（+15.0%）

Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 36

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Haoyu WANG, Zexu ZHANG, Shan WEN, Jinlong LIU, Beixiao ZHU, Weimin BAO. Task allocation algorithm for UAV swarm based on temporal coupling analysis [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(2): 332075-332075.
[2]	Pan ZHOU, Ni LI, Jiangtao HUANG, Qinglin YANG, Yunxiao LIAN. Autonomous decision-making in close-range game under imperfect information for unmanned aerial vehicles [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(S1): 732215-732215.
[3]	Rongzu LI, Li LIU, Dun YANG. Optimal design of hydrogen-powered UAV based on multi-source domain fusion surrogate model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 630979-630979.
[4]	Kaifang WAN, Zhilin WU, Yunhui WU, Haozhi QIANG, Yibo WU, Bo LI. Cooperative location of multiple UAVs with deep reinforcement learning in GPS-denied environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331024-331024.
[5]	Lingfeng JIANG, Xinkai LI, Hai ZHANG, Hanwei LI, Hongli ZHANG. Mapless navigation of UAVs in dynamic environments based on an improved TD3 algorithm [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331035-331035.
[6]	Fujie WU, Bowen WANG, Jingya QI, Mingzhi CAO, Yingjun SANG, Sheng LI, Yuzhen ZHANG, Qian CHEN, Chao ZUO. A review of airborne multi-aperture panoramic image compositing [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630505-630505.
[7]	Min YANG, Guanjun LIU, Ziyuan ZHOU. Control of lunar landers based on secure reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630553-630553.
[8]	Yiquan WU, Kang TONG. Research advances on deep learning-based small object detection in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 30848-030848.
[9]	Qichao XIE, Chengyu CAO, Yiyun ZHAO, Fanbiao LI. Integrated guidance and control method based on deep reinforcement learning parameter tuning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(24): 632345-632345.
[10]	Guocheng YAN, Honglun WANG, Yanxiang WANG, Yuebin LUN, Junfan ZHU. Prescribed performance anti-swing control for wing rotation process of UAV towed aerial recovery [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(24): 331840-331840.
[11]	Wei HUANG, Jiahao PAN, Chu HE. Wavelet time-frequency localization-based model compression for UAV object detection [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(23): 631952-631952.
[12]	Anping ZHANG, Hao DONG. UAV swarms and their takeoff method for high-end warfare [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(22): 331034-331034.
[13]	Yahang SONG, Xin ZHANG, Zhiming MA, Zhengyu ZUO. Airfoil gust alleviation using a plasma actuator in low-speed wind tunnel test [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(22): 131975-131975.
[14]	Yicheng SONG, Ruiyun QI, Bin JIANG. Distributed topology reconstruction of UAV formation network under communication fault [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(22): 331914-331914.
[15]	Chen WANG, Caisheng WEI, Zeyang YIN, Kai JIN, Xingchen LI. Collaborative planning of multi-UAV trajectories and communication strategies considering channel resource constraints [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 331837-331837.