电子电气工程与控制

基于马尔可夫决策的四轴飞行器自动着陆方法

  • 顾兆军 ,
  • 赵欢 ,
  • 王家亮 ,
  • 聂留阳
展开
  • 1.中国民航大学 信息安全测评中心,天津 300300
    2.中国民航大学 计算机科学与技术学院,天津 300300

收稿日期: 2023-09-26

  修回日期: 2023-11-05

  录用日期: 2023-12-27

  网络出版日期: 2024-01-11

基金资助

民航安全能力建设资金(PESA2022093);中国民航大学信息安全测评中心开放基金(ISECCA-202007);中国民航大学研究生科技创新基金(2022YJS064)

Automatic landing method for quad-rotor helicopter based on Markov decision process

  • Zhaojun GU ,
  • Huan ZHAO ,
  • Jialiang WANG ,
  • Liuyang NIE
Expand
  • 1.Information Security Evaluation Center,Civil Aviation University of China,Tianjin 300300,China
    2.College of Computer Science and Technology,Civil Aviation University of China,Tianjin 300300,China

Received date: 2023-09-26

  Revised date: 2023-11-05

  Accepted date: 2023-12-27

  Online published: 2024-01-11

Supported by

Civil Aviation Safety Capacity Building Funding(PESA2022093);Open Fund of Information Security Evaluation Center of Civil Aviation University of China(ISECCA-202007);Graduate Science and Technology Innovation Fund of Civil Aviation University of China(2022YJS064)

摘要

针对仅使用民用GPS引导四轴飞行器自动着陆时偏差较大的问题,基于马尔可夫决策设计了一种能够有效提升自动着陆精度的引导方法。首先,设计一种基于改进人工势场算法的对准方法,使用AprilTag3识别算法快速识别着陆标签,对标签中心划定势场,使飞行器可在力学牵引下进行偏航对准和云台俯仰对准;其次,将飞行器的着陆过程建立为马尔可夫决策模型,设置动作集、状态集与奖励集之间的状态转移关系,防止飞行器偏离预期状态;最后,结合飞行器自动着陆的问题环境,在传统马尔可夫决策模型的基础上增加信息集,利用飞行信息辅助转移关系的判定,提高状态与动作的决策准确性。通过安娜飞(ANAFI)四轴飞行器平台进行实际飞行验证,实验结果表明,基于马尔可夫决策模型的自动着陆引导方法能够有效抑制飞行器在着陆过程中的过度动作和误判断,着陆误差由米级缩小至约10 cm,满足四轴飞行器自动着陆的精度要求。

本文引用格式

顾兆军 , 赵欢 , 王家亮 , 聂留阳 . 基于马尔可夫决策的四轴飞行器自动着陆方法[J]. 航空学报, 2024 , 45(15) : 329652 -329652 . DOI: 10.7527/S1000-6893.2023.29652

Abstract

When using only civilian GPS guidance for automatic landing of quad-rotor helicopters, there exists the problem of significant deviation. A guidance method based on Markov decision is designed to effectively improve the accuracy of automatic landing. Firstly, an alignment method based on the improved artificial potential field algorithm is designed. The AprilTag3 recognition algorithm is used to quickly identify the landing tag and build the potential field at its center. The yaw alignment of aircraft and pitch alignment of gimbal are performed under the gravitational pull. Secondly, the landing process is established as a Markov Decision Process (MDP) model. The state transition relationship among the action set, state set, and reward set is designed to prevent the aircraft from deviating from the expected state. Finally, considering the environment of automatic landing, an information set is added to the MDP model. The information during flight is used to assist in the determination of the transition relationship, so as to improve the decision-making accuracy of states and actions. Actual flight verification is carried out on the ANAFI quad-rotor helicopter platform. The experimental results show that the automatic landing guidance method based on the MDP model can effectively suppress excessive movements and misjudgments of the aircraft during the landing process. The landing error has been reduced from the meter level to about 10 centimeters, meeting the accuracy requirements for automatic landing of quad-rotor helicopters.

参考文献

1 张良阳, 李占科, 韩海洋. 微型无人机栖息设计技术综述[J]. 航空学报202344(12): 027573.
  ZHANG L Y, LI Z K, HAN H Y. A review of perching technology of micro-UAV[J]. Acta Aeronautica et Astronautica Sinica202344(12): 027573 (in Chinese).
2 SONG H Y, YU J C, QIU J T, et al. Multi-UAV disaster environment coverage planning with limited-endurance[C]∥ 2022 International Conference on Robotics and Automation. Piscataway: IEEE Press, 2022: 10760-10766.
3 ZHU X N. Analysis of military application of UAV swarm technology[C]∥ 2020 3rd International Conference on Unmanned Systems. Piscataway: IEEE Press, 2020: 1200-1204.
4 SAH B, GUPTA R, BANI-HANI D. Analysis of barriers to implement drone logistics[J]. International Journal of Logistics Research and Applications202124(6): 531-550.
5 MERHEB A R, NOURA H, BATEMAN F. Emergency control of AR drone quadrotor UAV suffering a total loss of one rotor[J]. IEEE/ASME Transactions on Mechatronics201722(2): 961-971.
6 倪静, 马波, 杨朝旭, 等. 视觉/惯性着陆组合引导方法设计与试验[J]. 航空学报202344(S1): 727636.
  NI J, MA B, YANG Z X, et al. Design and test of visual-inertial integrated method for landing guidance[J]. Acta Aeronautica et Astronautica Sinica202344(S1): 727636 (in Chinese).
7 SUPRIYONO H, Design AKHARA A., building and performance testing of GPS and computer vision combination for increasing landing precision of quad-copter drone[J]. Journal of Physics: Conference Series20211858(1): 012074.
8 TRUONG N Q, NGUYEN P H, NAM S H, et al. Deep learning-based super-resolution reconstruction and marker detection for drone landing[J]. IEEE Access20197: 61639-61655.
9 NEPAL U, ESLAMIAT H. Comparing YOLOv3, YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs[J]. Sensors202222(2): 464.
10 唐进, 梁彦刚, 白志会, 等. 基于DQN的旋翼无人机着陆控制算法[J]. 系统工程与电子技术202345(5): 1451-1460.
  TANG J, LIANG Y G, BAI Z H, et al. Landing control algorithm of rotor UAV based on DQN[J]. Systems Engineering and Electronics202345(5): 1451-1460 (in Chinese).
11 NIU G C, YANG Q K, GAO Y F, et al. Vision-based autonomous landing for unmanned aerial and ground vehicles cooperative systems[J]. IEEE Robotics and Automation Letters20227(3): 6234-6241.
12 赵燕伟, 张健, 周仙明, 等. 基于视觉-磁引导的无人机动态跟踪与精准着陆[J]. 浙江大学学报(工学版)202155(1): 96-108.
  ZHAO Y W, ZHANG J, ZHOU X M, et al. Dynamic tracking and precise landing of UAV based on visual magnetic guidance[J]. Journal of Zhejiang University (Engineering Science)202155(1): 96-108 (in Chinese).
13 刘晨阳, 吴大伟, 郭一泽, 等. 不确定强耦合下四旋翼姿态鲁棒自适应控制[J]. 航空学报202344(S1): 727645.
  LIU C Y, WU D W, GUO Y Z, et al. Robust adaptive attitude control of quadrotor with uncertain strong coupling[J]. Acta Aeronautica et Astronautica Sinica202344(S1): 727645 (in Chinese).
14 KALLWIES J, FORKEL B, WUENSCHE H J. Determining and improving the localization accuracy of AprilTag detection[C]∥ 2020 IEEE International Conference on Robotics and Automation. Piscataway: IEEE Press, 2020: 8288-8294.
15 KALAITZAKIS M, CAIN B, CARROLL S, et al. Fiducial markers for pose estimation: Overview, applications and experimental comparison of the ARTag, AprilTag, ArUco and STag markers[J]. Journal of Intelligent & Robotic Systems2021101(4): 1-26.
16 王羿, 叶辉, 杨晓飞. 基于无源性与势场法的四旋翼避障与位置控制[J]. 航空学报202344(S1): 727492.
  WANG Y, YE H, YANG X F. A position control and obstacle avoidance method for quadrotor via approach based on passivity and artificial potential filed[J]. Acta Aeronautica et Astronautica Sinica202344(S1): 727492 (in Chinese).
17 OROZCO-ROSAS U, MONTIEL O, SEPúLVEDA R. Mobile robot path planning using membrane evolutionary artificial potential field[J]. Applied Soft Computing201977: 236-251.
18 FENG Z K, HUANG M X, WU D, et al. Multi-agent reinforcement learning with policy clipping and average evaluation for UAV-assisted communication Markov game[J]. IEEE Transactions on Intelligent Transportation Systems202324(12): 14281-14293.
19 ROY V. Convergence diagnostics for Markov chain Monte Carlo[J]. Annual Review of Statistics and Its Application20207: 387-412.
20 KALLUS N, UEHARA M. Double reinforcement learning for efficient off-policy evaluation in Markov decision processes[J]. The Journal of Machine Learning Research202021(1): 6742-6804.
21 PARK K, NISHIYAMA M, NAKADA N, et al. Effect of the martensite distribution on the strain hardening and ductile fracture behaviors in dual-phase steel[J]. Materials Science and Engineering: A2014604: 135-141.
22 GUSSEV M N, BUSBY J T, BYUN T S, et al. Twinning and martensitic transformations in nickel-enriched 304 austenitic steel during tensile and indentation deformations[J]. Materials Science and Engineering: A2013588: 299-307.
23 JUAREZ-SALAZAR R, ZHENG J, DIAZ-RAMIREZ V H. Distorted pinhole camera modeling and calibration[J]. Applied Optics202059(36): 11310-11318.
24 RODRIGUEZ-RAMOS A, SAMPEDRO C, BAVLE H, et al. A deep reinforcement learning strategy for UAV autonomous landing on a moving platform[J]. Journal of Intelligent & Robotic Systems201993(1/2): 351-366.
25 BADAKIS G, KOUTSOUBELIAS M, LALIS S. Robust precision landing for autonomous drones combining vision-based and infrared sensors[C]∥ 2021 IEEE Sensors Applications Symposium. Piscataway: IEEE Press, 2021: 1-6.
26 LI Z, CHEN Y, LU H, et al. UAV autonomous landing technology based on AprilTags vision positioning algorithm[C]∥ 2019 Chinese Control Conference. Piscataway: IEEE Press, 2019: 8148-8153.
文章导航

/