收稿日期: 2023-05-29
修回日期: 2023-07-12
录用日期: 2023-11-06
网络出版日期: 2023-11-16
基金资助
电子信息系统复杂电磁环境效应国家重点实验室资助课题(CEMEE2021K0101A)
Autonomous trajectory design for IoT data collection by UAV
Received date: 2023-05-29
Revised date: 2023-07-12
Accepted date: 2023-11-06
Online published: 2023-11-16
Supported by
State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System(CEMEE2021K0101A)
路径规划旨在为无人机（UAV）提供安全可靠的行进路径，而实际环境的动态性极大地增加了路径规划的难度。本文针对物联网（IoT）节点处的数据收集问题，构建了一个复杂的3D动态环境，在多评论家深度确定性梯度算法（MCDDPG）的基础上提出一种基于无人机电量约束、路径长度最小化（MCDDPG-EPM）算法。算法考虑无人机自身电量约束及其在物联网节点间的调度问题，确保无人机在电量供应安全的前提下以较短的路径长度完成数据采集工作。特别地，为了应对动态环境下突发障碍物移动问题，提出信息增强的概念，以降低移动障碍物带来的路径不确定性。仿真结果表明，当物联网节点数为20时，所提算法相较于双延迟深度确定性策略梯度算法（TD3）、传统A*算法和蚁群算法（ACO）分别节省了11.8%、13.2%和15.1%的电量消耗。
关键词： 路径规划; 物联网; 无人机; 多评论家深度确定性梯度算法; 双延迟深度确定性策略梯度算法
张薇 , 何若俊 . 面向物联网数据收集的无人机自主路径规划[J]. 航空学报, 2024 , 45(8) : 329054 -329054-1 . DOI: 10.7527/S1000-6893.2023.29054
Trajectory design is critical for navigation of Unmanned Aerial Vehicle （UAV） as it ensures the provision of safe and reliable travel paths. However， the dynamics of the actual environment significantly amplify the challenges associated with path planning. To address this issue， we design a complex 3D dynamic environment， in which a UAV is employed to collect data from multiple ground nodes of Internet of Things （IoT）. Drawing inspiration from the Multi-Critic Deep Deterministic Policy Gradient （MCDDPG）， we propose a Multi-Critic DDPG-Energy constrained Path-length-Minimization （MCDDPG-EPM） algorithm， which considers the UAV’s energy constraints and related node scheduling problems， ensuring that the data can be completely collected with short path length and sufficient power. In particular， an information enhancement method is introduced to handle the unpredictability of moving obstacles in dynamic environments. Simulation results demonstrate that compared with the Twin Delayed Deep Deterministic policy gradient algorithm （TD3）， traditional A* algorithm， and the traditional Ant Colony Optimization （ACO）， the proposed approach achieves energy savings of 11.8%， 13.2%， and 15.1%， respectively， when the number of the IoT nodes is 20.
| 1 | ZENG Y， ZHANG R， LIM T J. Wireless communications with unmanned aerial vehicles： Opportunities and challenges［J］. IEEE Communications Magazine， 2016， 54（5）： 36-42. |
| 2 | LI B， FEI Z S， ZHANG Y. UAV communications for 5G and beyond： Recent advances and future trends［J］. IEEE Internet of Things Journal， 2019， 6（2）： 2241-2263. |
| 3 | WU Q Q， XU J， ZENG Y， et al. A comprehensive overview on 5G-and-beyond networks with UAVs： From communications to sensing and intelligence［J］. IEEE Journal on Selected Areas in Communications， 2021， 39（10）： 2912-2945. |
| 4 | ZANELLA A， BUI N， CASTELLANI A， et al. Internet of things for smart cities［J］. IEEE Internet of Things Journal， 2014， 1（1）： 22-32. |
| 5 | LI J X， ZHAO H T， WANG H J， et al. Joint optimization on trajectory， altitude， velocity， and link scheduling for minimum mission time in UAV-aided data collection［J］. IEEE Internet of Things Journal， 2020， 7（2）： 1464-1475. |
| 6 | LI X， TAN J W， LIU A F， et al. A novel UAV-enabled data collection scheme for intelligent transportation system through UAV speed control［J］. IEEE Transactions on Intelligent Transportation Systems， 2021， 22（4）： 2100-2110. |
| 7 | 胡敬， 田俊曦， 邹世明， 等. 基于自适应卡尔曼滤波的联合RSS/TOA/INS无人机定位算法［J］. 无人系统技术， 2022， 5（2）： 62-70. |
| HU J， TIAN J X， ZOU S M， et al. A hybrid RSS/TOA/INS UAV localization algorithm based on adaptive Kalman filter［J］. Unmanned Systems Technology， 2022， 5（2）： 62-70 （in Chinese）. | |
| 8 | SUNG Y， TOKEKAR P. GM-PHD filter for searching and tracking an unknown number of targets with a mobile sensor with limited FOV［J］. IEEE Transactions on Automation Science and Engineering， 2022， 19（3）： 2122-2134. |
| 9 | YANG X， DING M Y， ZHOU C P. Fast marine route planning for UAV using improved sparse A* algorithm［C］∥2010 Fourth International Conference on Genetic and Evolutionary Computing. Piscataway： IEEE Press， 2010： 190-193. |
| 10 | KALA R， WARWICK K. Planning of multiple autonomous vehicles using RRT［C］∥2011 IEEE 10th International Conference on Cybernetic Intelligent Systems （CIS）. Piscataway： IEEE Press， 2011： 20-25. |
| 11 | DORIGO M， MANIEZZO V， COLORNI A. Ant system： Optimization by a colony of cooperating agents［J］. IEEE Transactions on Systems， Man， and Cybernetics Part B， Cybernetics： A Publication of the IEEE Systems， Man， and Cybernetics Society， 1996， 26（1）： 29-41. |
| 12 | 周彬， 郭艳， 李宁， 等. 基于导向强化Q学习的无人机路径规划［J］. 航空学报， 2021， 42（9）： 325109. |
| ZHOU B， GUO Y， LI N， et al. Path planning of UAV using guided enhancement Q-learning algorithm［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（9）： 325109 （in Chinese）. | |
| 13 | MOON J， PAPAIOANNOU S， LAOUDIAS C， et al. Deep reinforcement learning multi-UAV trajectory control for target tracking［J］. IEEE Internet of Things Journal， 2021， 8（20）： 15441-15455. |
| 14 | WANG Y， GAO Z， ZHANG J， et al. Trajectory design for UAV-based Internet of Things data collection： A deep reinforcement learning approach［J］. IEEE Internet of Things Journal， 2022， 9（5）： 3899-3912. |
| 15 | SINGLA A， PADAKANDLA S， BHATNAGAR S. Memory-based deep reinforcement learning for obstacle avoidance in UAV with limited environment knowledge［J］. IEEE Transactions on Intelligent Transportation Systems， 2021， 22（1）： 107-118. |
| 16 | SHIRI H， SEO H， PARK J， et al. Attention-based communication and control for multi-UAV path planning［J］. IEEE Wireless Communications Letters， 2022， 11（7）： 1409-1413. |
| 17 | LI B H， HUANG Z L， CHEN T W， et al. MSN： Mapless short-range navigation based on time critical deep reinforcement learning［J］. IEEE Transactions on Intelligent Transportation Systems， 2023， 24（8）： 8628-8637. |
| 18 | XU S， ZHANG X Y， LI C G， et al. Deep reinforcement learning approach for joint trajectory design in multi-UAV IoT networks［J］. IEEE Transactions on Vehicular Technology， 2022， 71（3）： 3389-3394. |
| 19 | TANG X M， CHAI Y， LIU Q. A 2D UAV path planning method based on reinforcement learning in the presence of dense obstacles and kinematic constraints［C］∥2022 IEEE 11th Data Driven Control and Learning Systems Conference （DDCLS）. Piscataway： IEEE Press， 2022： 306-311. |
| 20 | KHAMIDEHI B， SOUSA E S. Reinforcement-learning-aided safe planning for aerial robots to collect data in dynamic environments［J］. IEEE Internet of Things Journal， 2022， 9（15）： 13901-13912. |
| 21 | AL-HOURANI A， KANDEEPAN S， LARDNER S. Optimal LAP altitude for maximum coverage［J］. IEEE Wireless Communications Letters， 2014， 3（6）： 569-572. |
| 22 | YAO J J， ANSARI N. QoS-aware power control in Internet of drones for data collection service［J］. IEEE Transactions on Vehicular Technology， 2019， 68（7）： 6649-6656. |
| 23 | ZHU B T， BEDEER E， NGUYEN H H， et al. UAV trajectory planning for AoI-minimal data collection in UAV-aided IoT networks by transformer［J］. IEEE Transactions on Wireless Communications， 2023， 22（2）： 1343-1358. |
| 24 | ZENG Y， XU J， ZHANG R. Energy minimization for wireless communication with rotary-wing UAV［J］. IEEE Transactions on Wireless Communications， 2019， 18（4）： 2329-2345. |
| 25 | TENG T H， TAN A H， ZURADA J M. Self-organizin-g neural networks integrating domain knowledge and reinforcement learning［J］. IEEE Transactions on Neur-al Networks and Learning Systems， 2015， 26（5）： 889-902. |
| 26 | LILLICRAP T P， HUNT J J， PRITZEL A， et al. Continuous control with deep reinforcement learning［DB/OL］. arXiv preprint： 1509.02971， 2015. |
| 27 | WU J， WANG R， LI R Y， et al. Multi-critic DDPG method and double experience replay［C］∥2018 IEEE International Conference on Systems， Man， and Cybernetics （SMC）. Piscataway： IEEE Press， 2018： 165-171. |
| 28 | PARDALOS P M. Convex optimization theory［J］. Optimization Methods & Software， 2010， 25（3）： 487. |
/
| 〈 |
|
〉 |
CJA