Target State Collaboration and Intelligent Perception

Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks

  • Lei ZHANG ,
  • Can TIAN ,
  • Fangqing WEN ,
  • Qinghe ZHANG ,
  • Han LIU
Expand
  • 1.Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering,China Three Gorges University,Yichang 443000,China
    2.College of Computer and Information Technology,China Three Gorges University,Yichang 443000,China

Received date: 2025-02-20

  Revised date: 2025-04-14

  Accepted date: 2025-05-06

  Online published: 2025-05-13

Supported by

National Natural Science Foundation of China(62271286);Open Fund From Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering(2024SDSJ02)

Abstract

The Mobile Edge Computing (MEC) network assisted by Unmanned Aerial Vehicles (UAV)demonstrates great potential in emergency response, real-time monitoring, and other fields. However, the efficient operation of MEC network encounters challenges stemming from multiple optimization objectives, such as high energy consumption and high latency. Therefore, a Multi-Objective Evolution with Deep Deterministic Policy Gradient (MOE-DDPG) algorithm for UAV-assisted MEC network optimization is introduced. Firstly, an integrated multi-objective optimization model is established to ensure comprehensive performance of the MEC network by minimizing latency and energy consumption while maximizing the number of completed UAV tasks. Secondly, a bidirectional selection strategy for weight vector and individual matching is proposed to address the difficulty of balancing various objectives in traditional Deep Deterministic Policy Gradient (DDPG) algorithms when dealing with multi-objective optimization problems, thereby significantly enhancing population diversity. Finally, by organically fusing the Multi-Objective Evolution (MOE) algorithm and DDPG algorithm, a novel MOE-DDPG algorithm framework is proposed, which can optimize the overall performance of the MEC network in real time. The experimental results show that the MOE-DDPG algorithm not only significantly improves the distribution and convergence of the Pareto solution set but also effectively reduces energy consumption, latency, and increases the number of completed tasks.

Cite this article

Lei ZHANG , Can TIAN , Fangqing WEN , Qinghe ZHANG , Han LIU . Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2026 , 47(3) : 631880 -631880 . DOI: 10.7527/S1000-6893.2025.31880

References

[1] CAO B Q, YE H F, LIU J X, et al. SMART: Cost-aware service migration path selection based on deep reinforcement learning[J]. IEEE Transactions on Intelligent Transportation Systems202425(9): 12421-12436.
[2] CHENG S Y, REN T, ZHANG H, et al. A Stackelberg-game-based framework for edge pricing and resource allocation in mobile edge computing[J]. IEEE Internet of Things Journal202411(11): 20514-20530.
[3] 李伟, 郭艳, 李宁, 等. 智能反射面辅助无人机移动边缘计算任务数据最大化方法[J]. 航空学报202344(19): 328486.
  LI W, GUO Y, LI N, et al. Intelligent reflector surface assisted UAV mobile edge computing task data maximization method[J]. Acta Aeronautica et Astronautica Sinica202344(19): 328486 (in Chinese).
[4] SHAH Z, JAVED U, NAEEM M, et al. Mobile edge computing (MEC)-enabled UAV placement and computation efficiency maximization in disaster scenario[J]. IEEE Transactions on Vehicular Technology202372(10): 13406-13416.
[5] ZHAI J H, BI J, YUAN H T, et al. Cost-minimized microservice migration with autoencoder-assisted evolution in hybrid cloud and edge computing systems[J]. IEEE Internet of Things Journal202411(24): 40951-40967.
[6] 李伟, 郭艳, 何明, 等. 满意度驱动下无人机移动边缘计算服务缓存和资源分配方法[J]. 航空学报202445(19): 330017.
  LI W, GUO Y, HE M, et al. Satisfaction-driven services caching and resource allocation for UAV mobile edge computing[J]. Acta Aeronautica et Astronautica Sinica202445(19): 330017 (in Chinese).
[7] HUI M, CHEN J, YANG L, et al. UAV-assisted mobile edge computing: Optimal design of UAV altitude and task offloading[J]. IEEE Transactions on Wireless Communications202423(10): 13633-13647.
[8] 屈毓锛, 秦蓁, 马靖豪, 等. 面向空地协同移动边缘计算的服务布置策略[J]. 计算机学报202245(4): 781-797.
  QU Y B, QIN Z, MA J H, et al. Service provisioning for air-ground collaborative mobile edge computing[J]. Chinese Journal of Computers202245(4): 781-797 (in Chinese).
[9] CAO L M, HUO T, LI S B, et al. Cost optimization in edge computing: A survey[J]. Artificial Intelligence Review202457(11): 312.
[10] CHAI Z Y, LIU X, LI Y L. A computation offloading algorithm based on multi-objective evolutionary optimization in mobile edge computing[J]. Engineering Applications of Artificial Intelligence2023121: 105966.
[11] CUI Y Y, ZHANG D G, ZHANG T, et al. A novel offloading scheduling method for mobile application in mobile edge computing[J]. Wireless Networks202228(6): 2345-2363.
[12] FU S, ZHOU F H, HU R Q Y. Resource allocation in a relay-aided mobile edge computing system[J]. IEEE Internet of Things Journal20229(23): 23659-23669.
[13] EJAZ M, GUI J S, ASIM M, et al. RL-planner: Reinforcement learning-enabled efficient path planning in multi-UAV MEC systems[J]. IEEE Transactions on Network and Service Management202421(3): 3317-3329.
[14] CHEN C, GONG S M, ZHANG W J, et al. DRL-based contract incentive for wireless-powered and UAV-assisted backscattering MEC system[J]. IEEE Transactions on Cloud Computing202412(1): 264-276.
[15] LI B, LIU Y F, TAN L, et al. Digital twin assisted task offloading for aerial edge computing and networks[J]. IEEE Transactions on Vehicular Technology202271(10): 10863-10877.
[16] WANG L Y, ZHANG G L. Joint service caching, resource allocation and computation offloading in three-tier cooperative mobile edge computing system[J]. IEEE Transactions on Network Science and Engineering202310(6): 3343-3353.
[17] LI M, YU F R, SI P B, et al. UAV-assisted data transmission in blockchain-enabled M2M communications with mobile edge computing[J]. IEEE Network202034(6): 242-249.
[18] ZHANG S X, CAO R Y. Multi-objective optimization for UAV-enabled wireless powered IoT networks: An LSTM-based deep reinforcement learning approach[J]. IEEE Communications Letters202226(12): 3019-3023.
[19] LI L P, GUAN W Q, ZHAO C, et al. Trajectory planning, phase shift design, and IoT devices association in flying-RIS-assisted mobile edge computing[J]. IEEE Internet of Things Journal202411(1): 147-157.
[20] ZHANG S, JIN H L, GUO P K. IRS-assisted energy efficient communication for UAV mobile edge computing[J]. Computer Networks2024246: 110387.
[21] AL-HILO A, SAMIR M, ELHATTAB M, et al. RIS-assisted UAV for timely data collection in IoT networks[J]. IEEE Systems Journal202317(1): 431-442.
[22] SHI J L, LI C Y, GUAN Y C, et al. Multi-UAV-assisted computation offloading in DT-based networks: A distributed deep reinforcement learning approach[J]. Computer Communications2023210: 217-228.
[23] ZHAO X, ZHAO T H, WANG F Y, et al. SAC-based UAV mobile edge computing for energy minimization and secure data transmission[J]. Ad Hoc Networks2024157: 103435.
[24] LI B, XIE W C, YE Y H, et al. FlexEdge: Digital twin-enabled task offloading for UAV-aided vehicular edge computing[J]. IEEE Transactions on Vehicular Technology202372(8): 11086-11091.
[25] JIANG W W, AI B, LI M S, et al. Aerial-IRSs-assisted energy-efficient task offloading and computing[J]. IEEE Internet of Things Journal202411(11): 20178-20193.
[26] YU Y, TANG J, HUANG J Y, et al. Multi-objective optimization for UAV-assisted wireless powered IoT networks based on extended DDPG algorithm[J]. IEEE Transactions on Communications202169(9): 6361-6374.
[27] ZHOU C H, WU W, HE H L, et al. Deep reinforcement learning for delay-oriented IoT task scheduling in SAGIN[J]. IEEE Transactions on Wireless Communications202120(2): 911-925.
[28] 季薇, 杨许鑫, 李飞, 等. 无人机辅助MEC系统中基于最优SIC顺序的能耗优化方案[J]. 通信学报202445(2): 18-30.
  JI W, YANG X X, LI F, et al. Energy consumption optimization scheme in UAV-assisted MEC system based on optimal SIC order[J]. Journal on Communications202445(2): 18-30 (in Chinese).
[29] YI X J, YU H Y, XU T. Solving multi-objective weapon-target assignment considering reliability by improved MOEA/D-AM2M[J]. Neurocomputing2024563: 126906.
[30] WANG P F, YANG H, HAN G J, et al. Decentralized navigation with heterogeneous federated reinforcement learning for UAV-enabled mobile edge computing[J]. IEEE Transactions on Mobile Computing202423(12): 13621-13638.
[31] HAO H, XU C Q, ZHANG W, et al. Joint task offloading, resource allocation, and trajectory design for multi-UAV cooperative edge computing with task priority[J]. IEEE Transactions on Mobile Computing202423(9): 8649-8663.
[32] PERVEZ F, SULTANA A, YANG C G, et al. Energy and latency efficient joint communication and computation optimization in a multi-UAV-assisted MEC network[J]. IEEE Transactions on Wireless Communications202423(3): 1728-1741.
[33] SONG F H, DENG M S, XING H L, et al. Energy-efficient trajectory optimization with wireless charging in UAV-assisted MEC based on multi-objective reinforcement learning[J]. IEEE Transactions on Mobile Computing202423(12): 10867-10884.
[34] JIN J L, XU Y J. Optimal policy characterization enhanced proximal policy optimization for multitask scheduling in cloud computing[J]. IEEE Internet of Things Journal20229(9): 6418-6433.
[35] SHEN S G, HAO X B, GAO Z J, et al. SAC-PP: Jointly optimizing privacy protection and computation offloading for mobile edge computing[J]. IEEE Transactions on Network and Service Management202421(6): 6190-6203.
[36] TAN L, GUO S T, ZHOU P Z, et al. HAT: Task offloading and resource allocation in RIS-assisted collaborative edge computing[J]. IEEE Transactions on Network Science and Engineering202411(5): 4665-4678.
Outlines

/