ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks
Received date: 2025-02-20
Revised date: 2025-04-14
Accepted date: 2025-05-06
Online published: 2025-05-13
Supported by
National Natural Science Foundation of China(62271286);Open Fund From Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering(2024SDSJ02)
The Mobile Edge Computing (MEC) network assisted by Unmanned Aerial Vehicles (UAV)demonstrates great potential in emergency response, real-time monitoring, and other fields. However, the efficient operation of MEC network encounters challenges stemming from multiple optimization objectives, such as high energy consumption and high latency. Therefore, a Multi-Objective Evolution with Deep Deterministic Policy Gradient (MOE-DDPG) algorithm for UAV-assisted MEC network optimization is introduced. Firstly, an integrated multi-objective optimization model is established to ensure comprehensive performance of the MEC network by minimizing latency and energy consumption while maximizing the number of completed UAV tasks. Secondly, a bidirectional selection strategy for weight vector and individual matching is proposed to address the difficulty of balancing various objectives in traditional Deep Deterministic Policy Gradient (DDPG) algorithms when dealing with multi-objective optimization problems, thereby significantly enhancing population diversity. Finally, by organically fusing the Multi-Objective Evolution (MOE) algorithm and DDPG algorithm, a novel MOE-DDPG algorithm framework is proposed, which can optimize the overall performance of the MEC network in real time. The experimental results show that the MOE-DDPG algorithm not only significantly improves the distribution and convergence of the Pareto solution set but also effectively reduces energy consumption, latency, and increases the number of completed tasks.
Lei ZHANG , Can TIAN , Fangqing WEN , Qinghe ZHANG , Han LIU . Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2026 , 47(3) : 631880 -631880 . DOI: 10.7527/S1000-6893.2025.31880
| [1] | CAO B Q, YE H F, LIU J X, et al. SMART: Cost-aware service migration path selection based on deep reinforcement learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(9): 12421-12436. |
| [2] | CHENG S Y, REN T, ZHANG H, et al. A Stackelberg-game-based framework for edge pricing and resource allocation in mobile edge computing[J]. IEEE Internet of Things Journal, 2024, 11(11): 20514-20530. |
| [3] | 李伟, 郭艳, 李宁, 等. 智能反射面辅助无人机移动边缘计算任务数据最大化方法[J]. 航空学报, 2023, 44(19): 328486. |
| LI W, GUO Y, LI N, et al. Intelligent reflector surface assisted UAV mobile edge computing task data maximization method[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(19): 328486 (in Chinese). | |
| [4] | SHAH Z, JAVED U, NAEEM M, et al. Mobile edge computing (MEC)-enabled UAV placement and computation efficiency maximization in disaster scenario[J]. IEEE Transactions on Vehicular Technology, 2023, 72(10): 13406-13416. |
| [5] | ZHAI J H, BI J, YUAN H T, et al. Cost-minimized microservice migration with autoencoder-assisted evolution in hybrid cloud and edge computing systems[J]. IEEE Internet of Things Journal, 2024, 11(24): 40951-40967. |
| [6] | 李伟, 郭艳, 何明, 等. 满意度驱动下无人机移动边缘计算服务缓存和资源分配方法[J]. 航空学报, 2024, 45(19): 330017. |
| LI W, GUO Y, HE M, et al. Satisfaction-driven services caching and resource allocation for UAV mobile edge computing[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(19): 330017 (in Chinese). | |
| [7] | HUI M, CHEN J, YANG L, et al. UAV-assisted mobile edge computing: Optimal design of UAV altitude and task offloading[J]. IEEE Transactions on Wireless Communications, 2024, 23(10): 13633-13647. |
| [8] | 屈毓锛, 秦蓁, 马靖豪, 等. 面向空地协同移动边缘计算的服务布置策略[J]. 计算机学报, 2022, 45(4): 781-797. |
| QU Y B, QIN Z, MA J H, et al. Service provisioning for air-ground collaborative mobile edge computing[J]. Chinese Journal of Computers, 2022, 45(4): 781-797 (in Chinese). | |
| [9] | CAO L M, HUO T, LI S B, et al. Cost optimization in edge computing: A survey[J]. Artificial Intelligence Review, 2024, 57(11): 312. |
| [10] | CHAI Z Y, LIU X, LI Y L. A computation offloading algorithm based on multi-objective evolutionary optimization in mobile edge computing[J]. Engineering Applications of Artificial Intelligence, 2023, 121: 105966. |
| [11] | CUI Y Y, ZHANG D G, ZHANG T, et al. A novel offloading scheduling method for mobile application in mobile edge computing[J]. Wireless Networks, 2022, 28(6): 2345-2363. |
| [12] | FU S, ZHOU F H, HU R Q Y. Resource allocation in a relay-aided mobile edge computing system[J]. IEEE Internet of Things Journal, 2022, 9(23): 23659-23669. |
| [13] | EJAZ M, GUI J S, ASIM M, et al. RL-planner: Reinforcement learning-enabled efficient path planning in multi-UAV MEC systems[J]. IEEE Transactions on Network and Service Management, 2024, 21(3): 3317-3329. |
| [14] | CHEN C, GONG S M, ZHANG W J, et al. DRL-based contract incentive for wireless-powered and UAV-assisted backscattering MEC system[J]. IEEE Transactions on Cloud Computing, 2024, 12(1): 264-276. |
| [15] | LI B, LIU Y F, TAN L, et al. Digital twin assisted task offloading for aerial edge computing and networks[J]. IEEE Transactions on Vehicular Technology, 2022, 71(10): 10863-10877. |
| [16] | WANG L Y, ZHANG G L. Joint service caching, resource allocation and computation offloading in three-tier cooperative mobile edge computing system[J]. IEEE Transactions on Network Science and Engineering, 2023, 10(6): 3343-3353. |
| [17] | LI M, YU F R, SI P B, et al. UAV-assisted data transmission in blockchain-enabled M2M communications with mobile edge computing[J]. IEEE Network, 2020, 34(6): 242-249. |
| [18] | ZHANG S X, CAO R Y. Multi-objective optimization for UAV-enabled wireless powered IoT networks: An LSTM-based deep reinforcement learning approach[J]. IEEE Communications Letters, 2022, 26(12): 3019-3023. |
| [19] | LI L P, GUAN W Q, ZHAO C, et al. Trajectory planning, phase shift design, and IoT devices association in flying-RIS-assisted mobile edge computing[J]. IEEE Internet of Things Journal, 2024, 11(1): 147-157. |
| [20] | ZHANG S, JIN H L, GUO P K. IRS-assisted energy efficient communication for UAV mobile edge computing[J]. Computer Networks, 2024, 246: 110387. |
| [21] | AL-HILO A, SAMIR M, ELHATTAB M, et al. RIS-assisted UAV for timely data collection in IoT networks[J]. IEEE Systems Journal, 2023, 17(1): 431-442. |
| [22] | SHI J L, LI C Y, GUAN Y C, et al. Multi-UAV-assisted computation offloading in DT-based networks: A distributed deep reinforcement learning approach[J]. Computer Communications, 2023, 210: 217-228. |
| [23] | ZHAO X, ZHAO T H, WANG F Y, et al. SAC-based UAV mobile edge computing for energy minimization and secure data transmission[J]. Ad Hoc Networks, 2024, 157: 103435. |
| [24] | LI B, XIE W C, YE Y H, et al. FlexEdge: Digital twin-enabled task offloading for UAV-aided vehicular edge computing[J]. IEEE Transactions on Vehicular Technology, 2023, 72(8): 11086-11091. |
| [25] | JIANG W W, AI B, LI M S, et al. Aerial-IRSs-assisted energy-efficient task offloading and computing[J]. IEEE Internet of Things Journal, 2024, 11(11): 20178-20193. |
| [26] | YU Y, TANG J, HUANG J Y, et al. Multi-objective optimization for UAV-assisted wireless powered IoT networks based on extended DDPG algorithm[J]. IEEE Transactions on Communications, 2021, 69(9): 6361-6374. |
| [27] | ZHOU C H, WU W, HE H L, et al. Deep reinforcement learning for delay-oriented IoT task scheduling in SAGIN[J]. IEEE Transactions on Wireless Communications, 2021, 20(2): 911-925. |
| [28] | 季薇, 杨许鑫, 李飞, 等. 无人机辅助MEC系统中基于最优SIC顺序的能耗优化方案[J]. 通信学报, 2024, 45(2): 18-30. |
| JI W, YANG X X, LI F, et al. Energy consumption optimization scheme in UAV-assisted MEC system based on optimal SIC order[J]. Journal on Communications, 2024, 45(2): 18-30 (in Chinese). | |
| [29] | YI X J, YU H Y, XU T. Solving multi-objective weapon-target assignment considering reliability by improved MOEA/D-AM2M[J]. Neurocomputing, 2024, 563: 126906. |
| [30] | WANG P F, YANG H, HAN G J, et al. Decentralized navigation with heterogeneous federated reinforcement learning for UAV-enabled mobile edge computing[J]. IEEE Transactions on Mobile Computing, 2024, 23(12): 13621-13638. |
| [31] | HAO H, XU C Q, ZHANG W, et al. Joint task offloading, resource allocation, and trajectory design for multi-UAV cooperative edge computing with task priority[J]. IEEE Transactions on Mobile Computing, 2024, 23(9): 8649-8663. |
| [32] | PERVEZ F, SULTANA A, YANG C G, et al. Energy and latency efficient joint communication and computation optimization in a multi-UAV-assisted MEC network[J]. IEEE Transactions on Wireless Communications, 2024, 23(3): 1728-1741. |
| [33] | SONG F H, DENG M S, XING H L, et al. Energy-efficient trajectory optimization with wireless charging in UAV-assisted MEC based on multi-objective reinforcement learning[J]. IEEE Transactions on Mobile Computing, 2024, 23(12): 10867-10884. |
| [34] | JIN J L, XU Y J. Optimal policy characterization enhanced proximal policy optimization for multitask scheduling in cloud computing[J]. IEEE Internet of Things Journal, 2022, 9(9): 6418-6433. |
| [35] | SHEN S G, HAO X B, GAO Z J, et al. SAC-PP: Jointly optimizing privacy protection and computation offloading for mobile edge computing[J]. IEEE Transactions on Network and Service Management, 2024, 21(6): 6190-6203. |
| [36] | TAN L, GUO S T, ZHOU P Z, et al. HAT: Task offloading and resource allocation in RIS-assisted collaborative edge computing[J]. IEEE Transactions on Network Science and Engineering, 2024, 11(5): 4665-4678. |
/
| 〈 |
|
〉 |