Abstract: Unmanned Aerial Vehicle (UAV) has been widely adopted to assist Wireless Sensor Networks (WSNs) in performing data collection tasks. However, time window constraints at the sensor nodes present new challenges. The UAV must not only ar-rive in the vicinity of each data-uploading node within its designated time window, but also complete the data collection task before the window closes. Inefficient trajectory planning increases the UAV’s flight distance, which may compromise the completeness of data collection. Although increasing flight speed can shorten travel time, it also accelerates energy depletion, potentially leading to task failure. To address these problems, We formulate a mathematical model for the UAV trajectory planning problem in a time-window-constrained complete data collection scenario, and then propose a reinforcement learn-ing framework based on a Hierarchical Hybrid Action Representation (H-HyAR) to jointly optimize the UAV's visiting order of target nodes, hovering offset, and flight speed, while capturing the hierarchical dependencies among these factors to min-imize the UAV’s flight distance during the data collection task. Numerous experiment results demonstrate that the H-HyAR algorithm outperforms three comparative hybrid action reinforcement learning algorithms and the Proximal Policy Optimiza-tion (PPO) algorithm in terms of flight distance and the influencing factors of this metric. Moreover, H-HyAR algorithm ex-hibits strong robustness and generalization capabilities.

Key words: unmanned aerial vehicle trajectory planning, hierarchical hybrid action representation, deep reinforcement learning, time window, complete data collection, wireless sensor networks