基于马尔可夫决策的四轴飞行器自动着陆方法

doi:10.7527/S1000-6893.2023.29652

Abstract

Abstract:

When using only civilian GPS guidance for automatic landing of quad-rotor helicopters， there exists the problem of significant deviation. A guidance method based on Markov decision is designed to effectively improve the accuracy of automatic landing. Firstly， an alignment method based on the improved artificial potential field algorithm is designed. The AprilTag3 recognition algorithm is used to quickly identify the landing tag and build the potential field at its center. The yaw alignment of aircraft and pitch alignment of gimbal are performed under the gravitational pull. Secondly， the landing process is established as a Markov Decision Process （MDP） model. The state transition relationship among the action set， state set， and reward set is designed to prevent the aircraft from deviating from the expected state. Finally， considering the environment of automatic landing， an information set is added to the MDP model. The information during flight is used to assist in the determination of the transition relationship， so as to improve the decision-making accuracy of states and actions. Actual flight verification is carried out on the ANAFI quad-rotor helicopter platform. The experimental results show that the automatic landing guidance method based on the MDP model can effectively suppress excessive movements and misjudgments of the aircraft during the landing process. The landing error has been reduced from the meter level to about 10 centimeters， meeting the accuracy requirements for automatic landing of quad-rotor helicopters.

Key words: Markov Decision Process (MDP), automatic landing, data fusion, artificial potential field, quad-rotor helicopter

CLC Number:

V279⁺.2

Zhaojun GU, Huan ZHAO, Jialiang WANG, Liuyang NIE. Automatic landing method for quad-rotor helicopter based on Markov decision process[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(15): 329652-329652.

Figures/Tables 19

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Table 1

Table 2

References 26

1	张良阳，李占科，韩海洋. 微型无人机栖息设计技术综述［J］. 航空学报， 2023， 44（12）： 027573.
	ZHANG L Y， LI Z K， HAN H Y. A review of perching technology of micro-UAV［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（12）： 027573 （in Chinese）.
2	SONG H Y， YU J C， QIU J T， et al. Multi-UAV disaster environment coverage planning with limited-endurance［C］∥ 2022 International Conference on Robotics and Automation. Piscataway： IEEE Press， 2022： 10760-10766.
3	ZHU X N. Analysis of military application of UAV swarm technology［C］∥ 2020 3rd International Conference on Unmanned Systems. Piscataway： IEEE Press， 2020： 1200-1204.
4	SAH B， GUPTA R， BANI-HANI D. Analysis of barriers to implement drone logistics［J］. International Journal of Logistics Research and Applications， 2021， 24（6）： 531-550.
5	MERHEB A R， NOURA H， BATEMAN F. Emergency control of AR drone quadrotor UAV suffering a total loss of one rotor［J］. IEEE/ASME Transactions on Mechatronics， 2017， 22（2）： 961-971.
6	倪静，马波，杨朝旭，等. 视觉/惯性着陆组合引导方法设计与试验［J］. 航空学报， 2023， 44（S1）： 727636.
	NI J， MA B， YANG Z X， et al. Design and test of visual-inertial integrated method for landing guidance［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（S1）： 727636 （in Chinese）.
7	SUPRIYONO H， Design AKHARA A.， building and performance testing of GPS and computer vision combination for increasing landing precision of quad-copter drone［J］. Journal of Physics： Conference Series， 2021， 1858（1）： 012074.
8	TRUONG N Q， NGUYEN P H， NAM S H， et al. Deep learning-based super-resolution reconstruction and marker detection for drone landing［J］. IEEE Access， 2019， 7： 61639-61655.
9	NEPAL U， ESLAMIAT H. Comparing YOLOv3， YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs［J］. Sensors， 2022， 22（2）： 464.
10	唐进，梁彦刚，白志会，等. 基于DQN的旋翼无人机着陆控制算法［J］. 系统工程与电子技术， 2023， 45（5）： 1451-1460.
	TANG J， LIANG Y G， BAI Z H， et al. Landing control algorithm of rotor UAV based on DQN［J］. Systems Engineering and Electronics， 2023， 45（5）： 1451-1460 （in Chinese）.
11	NIU G C， YANG Q K， GAO Y F， et al. Vision-based autonomous landing for unmanned aerial and ground vehicles cooperative systems［J］. IEEE Robotics and Automation Letters， 2022， 7（3）： 6234-6241.
12	赵燕伟，张健，周仙明，等. 基于视觉-磁引导的无人机动态跟踪与精准着陆［J］. 浙江大学学报（工学版）， 2021， 55（1）： 96-108.
	ZHAO Y W， ZHANG J， ZHOU X M， et al. Dynamic tracking and precise landing of UAV based on visual magnetic guidance［J］. Journal of Zhejiang University （Engineering Science）， 2021， 55（1）： 96-108 （in Chinese）.
13	刘晨阳，吴大伟，郭一泽，等. 不确定强耦合下四旋翼姿态鲁棒自适应控制［J］. 航空学报， 2023， 44（S1）： 727645.
	LIU C Y， WU D W， GUO Y Z， et al. Robust adaptive attitude control of quadrotor with uncertain strong coupling［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（S1）： 727645 （in Chinese）.
14	KALLWIES J， FORKEL B， WUENSCHE H J. Determining and improving the localization accuracy of AprilTag detection［C］∥ 2020 IEEE International Conference on Robotics and Automation. Piscataway： IEEE Press， 2020： 8288-8294.
15	KALAITZAKIS M， CAIN B， CARROLL S， et al. Fiducial markers for pose estimation： Overview， applications and experimental comparison of the ARTag， AprilTag， ArUco and STag markers［J］. Journal of Intelligent & Robotic Systems， 2021， 101（4）： 1-26.
16	王羿，叶辉，杨晓飞. 基于无源性与势场法的四旋翼避障与位置控制［J］. 航空学报， 2023， 44（S1）： 727492.
	WANG Y， YE H， YANG X F. A position control and obstacle avoidance method for quadrotor via approach based on passivity and artificial potential filed［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（S1）： 727492 （in Chinese）.
17	OROZCO-ROSAS U， MONTIEL O， SEPÚLVEDA R. Mobile robot path planning using membrane evolutionary artificial potential field［J］. Applied Soft Computing， 2019， 77： 236-251.
18	FENG Z K， HUANG M X， WU D， et al. Multi-agent reinforcement learning with policy clipping and average evaluation for UAV-assisted communication Markov game［J］. IEEE Transactions on Intelligent Transportation Systems， 2023， 24（12）： 14281-14293.
19	ROY V. Convergence diagnostics for Markov chain Monte Carlo［J］. Annual Review of Statistics and Its Application， 2020， 7： 387-412.
20	KALLUS N， UEHARA M. Double reinforcement learning for efficient off-policy evaluation in Markov decision processes［J］. The Journal of Machine Learning Research， 2020， 21（1）： 6742-6804.
21	PARK K， NISHIYAMA M， NAKADA N， et al. Effect of the martensite distribution on the strain hardening and ductile fracture behaviors in dual-phase steel［J］. Materials Science and Engineering： A， 2014， 604： 135-141.
22	GUSSEV M N， BUSBY J T， BYUN T S， et al. Twinning and martensitic transformations in nickel-enriched 304 austenitic steel during tensile and indentation deformations［J］. Materials Science and Engineering： A， 2013， 588： 299-307.
23	JUAREZ-SALAZAR R， ZHENG J， DIAZ-RAMIREZ V H. Distorted pinhole camera modeling and calibration［J］. Applied Optics， 2020， 59（36）： 11310-11318.
24	RODRIGUEZ-RAMOS A， SAMPEDRO C， BAVLE H， et al. A deep reinforcement learning strategy for UAV autonomous landing on a moving platform［J］. Journal of Intelligent & Robotic Systems， 2019， 93（1/2）： 351-366.
25	BADAKIS G， KOUTSOUBELIAS M， LALIS S. Robust precision landing for autonomous drones combining vision-based and infrared sensors［C］∥ 2021 IEEE Sensors Applications Symposium. Piscataway： IEEE Press， 2021： 1-6.
26	LI Z， CHEN Y， LU H， et al. UAV autonomous landing technology based on AprilTags vision positioning algorithm［C］∥ 2019 Chinese Control Conference. Piscataway： IEEE Press， 2019： 8148-8153.

实验情景	着陆结果	横向偏差 Δx/cm	纵向偏差 Δy/cm	平均直线偏差/cm
GPS着陆	结果1	52.8	120.6	100.49
	结果2	30.6	87.2
	结果3	48.7	60.2
MDP着陆	结果1	5.8	11.7	10.24
	结果2	7.1	8.8
	结果3	3.5	5.3

方法	硬件要求	着陆耗时/s	位置精度/cm	环境适应性
文献［24］：使用深度强化学习策略引导飞行器着陆，但需要进行复杂的训练，且因飞行器自身原因存在高度突变的错误估计情况	图形处理器	11.94~19.28	< 20	特定平台着陆
文献［25］：使用视觉-红外信标联合引导着陆，但需要搭载红外信标探测器，且要求着陆平台带有嵌套红外信标的分形标记，成本较高	红外信标探测器	17.50~41.50	< 25	红外信标着陆
文献［12］：使用磁源检测技术实现较高精度着陆，但需要使用高精度磁源检测器，且需要在平台上设置磁源生成器引导着陆，成本较高	高精度磁源检测器	25.00~27.00	< 5	磁源平台着陆
文献［26］：通过在着陆平台上设置不同尺寸的AprilTag标签，辅助飞行器解算不同高度下的姿态信息，但着陆精度波动稍大，标签布局复杂	单目相机	12.50~15.00	< 50	室外靶标着陆
所提方法：使用单一AprilTag标签作为着陆平台，设计基于数据融合的时齐性MDP引导飞行器自动着陆，环境适应性强，着陆精度较好	单目相机	15.60~24.80	< 11	室外靶标着陆

[1]	Hua YANG, Shusheng CHEN, Zhenghong GAO, Quanfeng JIANG, Wei ZHANG. Rotor aerodynamic data fusion based on Bayesian framework [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 128960-128960.
[2]	Kuo TIAN, Zhiyong SUN, Zengcong LI. High-precision digital twin method for structural static test monitoring [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(7): 429134-429134.
[3]	Hailang SONG, Jiandong ZHANG, Guoqing SHI, Qiming YANG, Yaozhong ZHANG. Comprehensive evaluation techniques and methods for flight test of avionics fire control system [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(5): 529687-529687.
[4]	Yi WANG, Hui YE, Xiaofei YANG. A position control and obstacle avoidance method for quadrotor via approach based on passivity and artificial potential filed [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S1): 727492-727492.
[5]	Huaijie ZHANG, Jingya MA, Haoyuan LIU, Pin GUO, Huichao DENG, Kun XU, Xilun DING. Indoor positioning technology of multi⁃rotor flying robot based on visual-inertial fusion [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(5): 426964-426964.
[6]	YANG Mingyue, SHOU Yingxin, TANG Yong, LIU Chang, XU Bin. Multi-quadrotor UAVs formation maintenance and collision avoidance control [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 726913-726913.
[7]	CAO Ming, WANG Peng, ZUO Hongfu, ZENG Haijun, SUN Jianzhong, YANG Weidong, WEI Fang, CHEN Xuefeng. Current status, challenges and opportunities of civil aero-engine diagnostics & health management Ⅱ: Comprehensive off-board diagnosis, life management and intelligent condition based MRO [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(9): 625574-625574.
[8]	LEI Shiying, SUN Jianzhong, LIU He. Cumulative damage index model and service reliability evaluation of turbine blade [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(3): 225064-225064.
[9]	ZHANG Weiwei, KOU Jiaqing, LIU Yilang. Prospect of artificial intelligence empowered fluid mechanics [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(4): 524689-524689.
[10]	WANG Wei, ZHAO Minrui, GAO Hongni, ZHU Shuai, QU Jue. Human-computer interaction: Intention recognition based on EEG and eye tracking [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(2): 324290-324290.
[11]	LI Xianqiang, MA Rong, ZHANG Shen, HOU Yanze, PEI Yifei. Improved design of ant colony algorithm and its application in path planning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2020, 41(S2): 724381-724381.
[12]	XU Bo, ZHANG Dalong. Tight formation flight control of UAVs based on pigeon inspired algorithm optimization by quantum behavior [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2020, 41(8): 323722-323722.
[13]	CHANG Qi, YANG Weixi, ZHAO Heng, MENG Yao, LIU Jun, GAO Heming. A multi-sensor based crack propagation monitoring research [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2020, 41(2): 223336-223336.
[14]	ZHANG Jing, HE You, PENG Yingning, LI Gang. Neural network and artificial potential field based cooperative and adversarial path planning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2019, 40(3): 322493-322493.
[15]	JIA Zhengrong, WANG Hangyu, LU Faxing. Improved circulating APF route planning based on obstacle convexification [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2019, 40(10): 323189-323189.

Automatic landing method for quad-rotor helicopter based on Markov decision process

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 19

References 26

Related Articles 15

Recommended Articles

Metrics

Comments