基于马尔可夫决策的四轴飞行器自动着陆方法
收稿日期: 2023-09-26
修回日期: 2023-11-05
录用日期: 2023-12-27
网络出版日期: 2024-01-11
基金资助
民航安全能力建设资金(PESA2022093);中国民航大学信息安全测评中心开放基金(ISECCA-202007);中国民航大学研究生科技创新基金(2022YJS064)
Automatic landing method for quad-rotor helicopter based on Markov decision process
Received date: 2023-09-26
Revised date: 2023-11-05
Accepted date: 2023-12-27
Online published: 2024-01-11
Supported by
Civil Aviation Safety Capacity Building Funding(PESA2022093);Open Fund of Information Security Evaluation Center of Civil Aviation University of China(ISECCA-202007);Graduate Science and Technology Innovation Fund of Civil Aviation University of China(2022YJS064)
针对仅使用民用GPS引导四轴飞行器自动着陆时偏差较大的问题,基于马尔可夫决策设计了一种能够有效提升自动着陆精度的引导方法。首先,设计一种基于改进人工势场算法的对准方法,使用AprilTag3识别算法快速识别着陆标签,对标签中心划定势场,使飞行器可在力学牵引下进行偏航对准和云台俯仰对准;其次,将飞行器的着陆过程建立为马尔可夫决策模型,设置动作集、状态集与奖励集之间的状态转移关系,防止飞行器偏离预期状态;最后,结合飞行器自动着陆的问题环境,在传统马尔可夫决策模型的基础上增加信息集,利用飞行信息辅助转移关系的判定,提高状态与动作的决策准确性。通过安娜飞(ANAFI)四轴飞行器平台进行实际飞行验证,实验结果表明,基于马尔可夫决策模型的自动着陆引导方法能够有效抑制飞行器在着陆过程中的过度动作和误判断,着陆误差由米级缩小至约10 cm,满足四轴飞行器自动着陆的精度要求。
顾兆军 , 赵欢 , 王家亮 , 聂留阳 . 基于马尔可夫决策的四轴飞行器自动着陆方法[J]. 航空学报, 2024 , 45(15) : 329652 -329652 . DOI: 10.7527/S1000-6893.2023.29652
When using only civilian GPS guidance for automatic landing of quad-rotor helicopters, there exists the problem of significant deviation. A guidance method based on Markov decision is designed to effectively improve the accuracy of automatic landing. Firstly, an alignment method based on the improved artificial potential field algorithm is designed. The AprilTag3 recognition algorithm is used to quickly identify the landing tag and build the potential field at its center. The yaw alignment of aircraft and pitch alignment of gimbal are performed under the gravitational pull. Secondly, the landing process is established as a Markov Decision Process (MDP) model. The state transition relationship among the action set, state set, and reward set is designed to prevent the aircraft from deviating from the expected state. Finally, considering the environment of automatic landing, an information set is added to the MDP model. The information during flight is used to assist in the determination of the transition relationship, so as to improve the decision-making accuracy of states and actions. Actual flight verification is carried out on the ANAFI quad-rotor helicopter platform. The experimental results show that the automatic landing guidance method based on the MDP model can effectively suppress excessive movements and misjudgments of the aircraft during the landing process. The landing error has been reduced from the meter level to about 10 centimeters, meeting the accuracy requirements for automatic landing of quad-rotor helicopters.
1 | 张良阳, 李占科, 韩海洋. 微型无人机栖息设计技术综述[J]. 航空学报, 2023, 44(12): 027573. |
ZHANG L Y, LI Z K, HAN H Y. A review of perching technology of micro-UAV[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(12): 027573 (in Chinese). | |
2 | SONG H Y, YU J C, QIU J T, et al. Multi-UAV disaster environment coverage planning with limited-endurance[C]∥ 2022 International Conference on Robotics and Automation. Piscataway: IEEE Press, 2022: 10760-10766. |
3 | ZHU X N. Analysis of military application of UAV swarm technology[C]∥ 2020 3rd International Conference on Unmanned Systems. Piscataway: IEEE Press, 2020: 1200-1204. |
4 | SAH B, GUPTA R, BANI-HANI D. Analysis of barriers to implement drone logistics[J]. International Journal of Logistics Research and Applications, 2021, 24(6): 531-550. |
5 | MERHEB A R, NOURA H, BATEMAN F. Emergency control of AR drone quadrotor UAV suffering a total loss of one rotor[J]. IEEE/ASME Transactions on Mechatronics, 2017, 22(2): 961-971. |
6 | 倪静, 马波, 杨朝旭, 等. 视觉/惯性着陆组合引导方法设计与试验[J]. 航空学报, 2023, 44(S1): 727636. |
NI J, MA B, YANG Z X, et al. Design and test of visual-inertial integrated method for landing guidance[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S1): 727636 (in Chinese). | |
7 | SUPRIYONO H, Design AKHARA A., building and performance testing of GPS and computer vision combination for increasing landing precision of quad-copter drone[J]. Journal of Physics: Conference Series, 2021, 1858(1): 012074. |
8 | TRUONG N Q, NGUYEN P H, NAM S H, et al. Deep learning-based super-resolution reconstruction and marker detection for drone landing[J]. IEEE Access, 2019, 7: 61639-61655. |
9 | NEPAL U, ESLAMIAT H. Comparing YOLOv3, YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs[J]. Sensors, 2022, 22(2): 464. |
10 | 唐进, 梁彦刚, 白志会, 等. 基于DQN的旋翼无人机着陆控制算法[J]. 系统工程与电子技术, 2023, 45(5): 1451-1460. |
TANG J, LIANG Y G, BAI Z H, et al. Landing control algorithm of rotor UAV based on DQN[J]. Systems Engineering and Electronics, 2023, 45(5): 1451-1460 (in Chinese). | |
11 | NIU G C, YANG Q K, GAO Y F, et al. Vision-based autonomous landing for unmanned aerial and ground vehicles cooperative systems[J]. IEEE Robotics and Automation Letters, 2022, 7(3): 6234-6241. |
12 | 赵燕伟, 张健, 周仙明, 等. 基于视觉-磁引导的无人机动态跟踪与精准着陆[J]. 浙江大学学报(工学版), 2021, 55(1): 96-108. |
ZHAO Y W, ZHANG J, ZHOU X M, et al. Dynamic tracking and precise landing of UAV based on visual magnetic guidance[J]. Journal of Zhejiang University (Engineering Science), 2021, 55(1): 96-108 (in Chinese). | |
13 | 刘晨阳, 吴大伟, 郭一泽, 等. 不确定强耦合下四旋翼姿态鲁棒自适应控制[J]. 航空学报, 2023, 44(S1): 727645. |
LIU C Y, WU D W, GUO Y Z, et al. Robust adaptive attitude control of quadrotor with uncertain strong coupling[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S1): 727645 (in Chinese). | |
14 | KALLWIES J, FORKEL B, WUENSCHE H J. Determining and improving the localization accuracy of AprilTag detection[C]∥ 2020 IEEE International Conference on Robotics and Automation. Piscataway: IEEE Press, 2020: 8288-8294. |
15 | KALAITZAKIS M, CAIN B, CARROLL S, et al. Fiducial markers for pose estimation: Overview, applications and experimental comparison of the ARTag, AprilTag, ArUco and STag markers[J]. Journal of Intelligent & Robotic Systems, 2021, 101(4): 1-26. |
16 | 王羿, 叶辉, 杨晓飞. 基于无源性与势场法的四旋翼避障与位置控制[J]. 航空学报, 2023, 44(S1): 727492. |
WANG Y, YE H, YANG X F. A position control and obstacle avoidance method for quadrotor via approach based on passivity and artificial potential filed[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S1): 727492 (in Chinese). | |
17 | OROZCO-ROSAS U, MONTIEL O, SEPúLVEDA R. Mobile robot path planning using membrane evolutionary artificial potential field[J]. Applied Soft Computing, 2019, 77: 236-251. |
18 | FENG Z K, HUANG M X, WU D, et al. Multi-agent reinforcement learning with policy clipping and average evaluation for UAV-assisted communication Markov game[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(12): 14281-14293. |
19 | ROY V. Convergence diagnostics for Markov chain Monte Carlo[J]. Annual Review of Statistics and Its Application, 2020, 7: 387-412. |
20 | KALLUS N, UEHARA M. Double reinforcement learning for efficient off-policy evaluation in Markov decision processes[J]. The Journal of Machine Learning Research, 2020, 21(1): 6742-6804. |
21 | PARK K, NISHIYAMA M, NAKADA N, et al. Effect of the martensite distribution on the strain hardening and ductile fracture behaviors in dual-phase steel[J]. Materials Science and Engineering: A, 2014, 604: 135-141. |
22 | GUSSEV M N, BUSBY J T, BYUN T S, et al. Twinning and martensitic transformations in nickel-enriched 304 austenitic steel during tensile and indentation deformations[J]. Materials Science and Engineering: A, 2013, 588: 299-307. |
23 | JUAREZ-SALAZAR R, ZHENG J, DIAZ-RAMIREZ V H. Distorted pinhole camera modeling and calibration[J]. Applied Optics, 2020, 59(36): 11310-11318. |
24 | RODRIGUEZ-RAMOS A, SAMPEDRO C, BAVLE H, et al. A deep reinforcement learning strategy for UAV autonomous landing on a moving platform[J]. Journal of Intelligent & Robotic Systems, 2019, 93(1/2): 351-366. |
25 | BADAKIS G, KOUTSOUBELIAS M, LALIS S. Robust precision landing for autonomous drones combining vision-based and infrared sensors[C]∥ 2021 IEEE Sensors Applications Symposium. Piscataway: IEEE Press, 2021: 1-6. |
26 | LI Z, CHEN Y, LU H, et al. UAV autonomous landing technology based on AprilTags vision positioning algorithm[C]∥ 2019 Chinese Control Conference. Piscataway: IEEE Press, 2019: 8148-8153. |
/
〈 |
|
〉 |