ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Automatic landing method for quad-rotor helicopter based on Markov decision process
Received date: 2023-09-26
Revised date: 2023-11-05
Accepted date: 2023-12-27
Online published: 2024-01-11
Supported by
Civil Aviation Safety Capacity Building Funding(PESA2022093);Open Fund of Information Security Evaluation Center of Civil Aviation University of China(ISECCA-202007);Graduate Science and Technology Innovation Fund of Civil Aviation University of China(2022YJS064)
When using only civilian GPS guidance for automatic landing of quad-rotor helicopters， there exists the problem of significant deviation. A guidance method based on Markov decision is designed to effectively improve the accuracy of automatic landing. Firstly， an alignment method based on the improved artificial potential field algorithm is designed. The AprilTag3 recognition algorithm is used to quickly identify the landing tag and build the potential field at its center. The yaw alignment of aircraft and pitch alignment of gimbal are performed under the gravitational pull. Secondly， the landing process is established as a Markov Decision Process （MDP） model. The state transition relationship among the action set， state set， and reward set is designed to prevent the aircraft from deviating from the expected state. Finally， considering the environment of automatic landing， an information set is added to the MDP model. The information during flight is used to assist in the determination of the transition relationship， so as to improve the decision-making accuracy of states and actions. Actual flight verification is carried out on the ANAFI quad-rotor helicopter platform. The experimental results show that the automatic landing guidance method based on the MDP model can effectively suppress excessive movements and misjudgments of the aircraft during the landing process. The landing error has been reduced from the meter level to about 10 centimeters， meeting the accuracy requirements for automatic landing of quad-rotor helicopters.
Zhaojun GU , Huan ZHAO , Jialiang WANG , Liuyang NIE . Automatic landing method for quad-rotor helicopter based on Markov decision process[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2024 , 45(15) : 329652 -329652 . DOI: 10.7527/S1000-6893.2023.29652
| 1 | 张良阳， 李占科， 韩海洋. 微型无人机栖息设计技术综述［J］. 航空学报， 2023， 44（12）： 027573. |
| ZHANG L Y， LI Z K， HAN H Y. A review of perching technology of micro-UAV［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（12）： 027573 （in Chinese）. | |
| 2 | SONG H Y， YU J C， QIU J T， et al. Multi-UAV disaster environment coverage planning with limited-endurance［C］∥ 2022 International Conference on Robotics and Automation. Piscataway： IEEE Press， 2022： 10760-10766. |
| 3 | ZHU X N. Analysis of military application of UAV swarm technology［C］∥ 2020 3rd International Conference on Unmanned Systems. Piscataway： IEEE Press， 2020： 1200-1204. |
| 4 | SAH B， GUPTA R， BANI-HANI D. Analysis of barriers to implement drone logistics［J］. International Journal of Logistics Research and Applications， 2021， 24（6）： 531-550. |
| 5 | MERHEB A R， NOURA H， BATEMAN F. Emergency control of AR drone quadrotor UAV suffering a total loss of one rotor［J］. IEEE/ASME Transactions on Mechatronics， 2017， 22（2）： 961-971. |
| 6 | 倪静， 马波， 杨朝旭， 等. 视觉/惯性着陆组合引导方法设计与试验［J］. 航空学报， 2023， 44（S1）： 727636. |
| NI J， MA B， YANG Z X， et al. Design and test of visual-inertial integrated method for landing guidance［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（S1）： 727636 （in Chinese）. | |
| 7 | SUPRIYONO H， Design AKHARA A.， building and performance testing of GPS and computer vision combination for increasing landing precision of quad-copter drone［J］. Journal of Physics： Conference Series， 2021， 1858（1）： 012074. |
| 8 | TRUONG N Q， NGUYEN P H， NAM S H， et al. Deep learning-based super-resolution reconstruction and marker detection for drone landing［J］. IEEE Access， 2019， 7： 61639-61655. |
| 9 | NEPAL U， ESLAMIAT H. Comparing YOLOv3， YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs［J］. Sensors， 2022， 22（2）： 464. |
| 10 | 唐进， 梁彦刚， 白志会， 等. 基于DQN的旋翼无人机着陆控制算法［J］. 系统工程与电子技术， 2023， 45（5）： 1451-1460. |
| TANG J， LIANG Y G， BAI Z H， et al. Landing control algorithm of rotor UAV based on DQN［J］. Systems Engineering and Electronics， 2023， 45（5）： 1451-1460 （in Chinese）. | |
| 11 | NIU G C， YANG Q K， GAO Y F， et al. Vision-based autonomous landing for unmanned aerial and ground vehicles cooperative systems［J］. IEEE Robotics and Automation Letters， 2022， 7（3）： 6234-6241. |
| 12 | 赵燕伟， 张健， 周仙明， 等. 基于视觉-磁引导的无人机动态跟踪与精准着陆［J］. 浙江大学学报（工学版）， 2021， 55（1）： 96-108. |
| ZHAO Y W， ZHANG J， ZHOU X M， et al. Dynamic tracking and precise landing of UAV based on visual magnetic guidance［J］. Journal of Zhejiang University （Engineering Science）， 2021， 55（1）： 96-108 （in Chinese）. | |
| 13 | 刘晨阳， 吴大伟， 郭一泽， 等. 不确定强耦合下四旋翼姿态鲁棒自适应控制［J］. 航空学报， 2023， 44（S1）： 727645. |
| LIU C Y， WU D W， GUO Y Z， et al. Robust adaptive attitude control of quadrotor with uncertain strong coupling［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（S1）： 727645 （in Chinese）. | |
| 14 | KALLWIES J， FORKEL B， WUENSCHE H J. Determining and improving the localization accuracy of AprilTag detection［C］∥ 2020 IEEE International Conference on Robotics and Automation. Piscataway： IEEE Press， 2020： 8288-8294. |
| 15 | KALAITZAKIS M， CAIN B， CARROLL S， et al. Fiducial markers for pose estimation： Overview， applications and experimental comparison of the ARTag， AprilTag， ArUco and STag markers［J］. Journal of Intelligent & Robotic Systems， 2021， 101（4）： 1-26. |
| 16 | 王羿， 叶辉， 杨晓飞. 基于无源性与势场法的四旋翼避障与位置控制［J］. 航空学报， 2023， 44（S1）： 727492. |
| WANG Y， YE H， YANG X F. A position control and obstacle avoidance method for quadrotor via approach based on passivity and artificial potential filed［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（S1）： 727492 （in Chinese）. | |
| 17 | OROZCO-ROSAS U， MONTIEL O， SEPúLVEDA R. Mobile robot path planning using membrane evolutionary artificial potential field［J］. Applied Soft Computing， 2019， 77： 236-251. |
| 18 | FENG Z K， HUANG M X， WU D， et al. Multi-agent reinforcement learning with policy clipping and average evaluation for UAV-assisted communication Markov game［J］. IEEE Transactions on Intelligent Transportation Systems， 2023， 24（12）： 14281-14293. |
| 19 | ROY V. Convergence diagnostics for Markov chain Monte Carlo［J］. Annual Review of Statistics and Its Application， 2020， 7： 387-412. |
| 20 | KALLUS N， UEHARA M. Double reinforcement learning for efficient off-policy evaluation in Markov decision processes［J］. The Journal of Machine Learning Research， 2020， 21（1）： 6742-6804. |
| 21 | PARK K， NISHIYAMA M， NAKADA N， et al. Effect of the martensite distribution on the strain hardening and ductile fracture behaviors in dual-phase steel［J］. Materials Science and Engineering： A， 2014， 604： 135-141. |
| 22 | GUSSEV M N， BUSBY J T， BYUN T S， et al. Twinning and martensitic transformations in nickel-enriched 304 austenitic steel during tensile and indentation deformations［J］. Materials Science and Engineering： A， 2013， 588： 299-307. |
| 23 | JUAREZ-SALAZAR R， ZHENG J， DIAZ-RAMIREZ V H. Distorted pinhole camera modeling and calibration［J］. Applied Optics， 2020， 59（36）： 11310-11318. |
| 24 | RODRIGUEZ-RAMOS A， SAMPEDRO C， BAVLE H， et al. A deep reinforcement learning strategy for UAV autonomous landing on a moving platform［J］. Journal of Intelligent & Robotic Systems， 2019， 93（1/2）： 351-366. |
| 25 | BADAKIS G， KOUTSOUBELIAS M， LALIS S. Robust precision landing for autonomous drones combining vision-based and infrared sensors［C］∥ 2021 IEEE Sensors Applications Symposium. Piscataway： IEEE Press， 2021： 1-6. |
| 26 | LI Z， CHEN Y， LU H， et al. UAV autonomous landing technology based on AprilTags vision positioning algorithm［C］∥ 2019 Chinese Control Conference. Piscataway： IEEE Press， 2019： 8148-8153. |
/
| 〈 |
|
〉 |
CJA