基于通道教师策略约束强化学习的无人机容错策略（飞行器协同作战技术专栏）

刘贞报; 王开; 贾真; 王潇; 颜伟俊

doi:10.7527/S1000-6893.2026.33358

航空学报 >

0 1 - 0

DOI: https://doi.org/10.7527/S1000-6893.2026.33358

基于通道教师策略约束强化学习的无人机容错策略（飞行器协同作战技术专栏）

刘贞报 ,
王开 ,
贾真 ,
王潇 ,
颜伟俊

展开

西北工业大学

收稿日期: 2026-01-13

修回日期: 2026-09-06

网络出版日期: 2026-09-10

基金资助

国家自然科学基金;中国博士后科学基金;CPSF博士后奖学金计划;陕西省博士后研究资助计划;中国航空科学基金

收起

Channel-Wise Teacher-Policy Constrained Reinforcement Learning for UAV Fault Tolerance Strategy

LIU Zhen-Bao ,
WANG Kai ,
JIA Zhen ,
WANG Xiao ,
YAN Wei-Jun

Expand

Received date: 2026-01-13

Revised date: 2026-09-06

Online published: 2026-09-10

Fold

摘要

渐进性电机退化故障对多旋翼无人机飞行安全构成严重威胁，为此，本文提出一种基于通道教师策略约束强化学习（CTCRL）的无人机容错策略。首先建立电机渐进退化故障模型以模拟故障动态响应；进而设计分通道控制分配算法，有效实现故障电机隔离与任务重构；在此基础上引入教师策略约束机制，以进一步增强故障下的控制重构能力。实验结果表明，在所设轻度故障下，CTCRL算法跟踪误差峰值为0.254 m，优于所对比的非坠毁算法；在所设中度故障下，CTCRL算法使故障电机平均温度相较于无温度约束对比算法降低最高达5.95 ℃；在所设重度故障下，CTCRL算法以199.76 J的最低触地动能坠落，仅为自由落体动能的14.6%，且在轨迹可控性上最优。所设硬件在环实验进一步验证该算法能够有效保证故障无人机在渐进性电机故障下的可靠性与鲁棒性，为集群编队任务持续能力提供底层技术支撑。

关键词： 容错控制; 控制分配; 多旋翼无人机; 强化学习; 电机故障

本文引用格式

刘贞报 , 王开 , 贾真 , 王潇 , 颜伟俊 . 基于通道教师策略约束强化学习的无人机容错策略（飞行器协同作战技术专栏）[J]. 航空学报, 0 : 1 -0 . DOI: 10.7527/S1000-6893.2026.33358

Abstract

Progressive motor degradation faults pose a serious threat to the flight safety of multi-rotor UAVs. To address this issue, this paper proposes a fault-tolerant control strategy based on Channel-wise Teacher-Policy Constrained Reinforcement Learning (CTCRL). First, a progressive motor degradation and temperature rise model is established to simulate the dynamic response of faults. Then, a channel-wise control allocation algorithm is designed to achieve effective isolation of faulty motors and mission reconfiguration. On this basis, a teacher policy constraint mechanism is introduced to further enhance the control reconfiguration capability under faults. Experimental results show that under the prescribed mild fault conditions, the CTCRL algorithm achieves a peak tracking error of 0.254 m, outperforming the compared non-crash algorithms. Under the prescribed moderate fault conditions, CTCRL reduces the average temperature of faulty motors by up to 5.95℃ compared to the contrast algorithm without temperature constraints. Under the prescribed severe fault conditions, CTCRL lands with the lowest impact kinetic energy of 199.76 J, which is only 14.6% of the free-fall kinetic energy, and exhibits the best trajectory controllability. The prescribed Hardware-in-the-loop experiments further verify that the proposed algorithm can effectively ensure the reliability and robustness of faulty UAV under progressive motor degradation, thereby providing underlying technical support for the mission sustainability of swarm formations.

Key words： Fault-Tolerant Control; Control allocation; Multi-rotor UAV; Reinforcement learning; Motor faults

参考文献

[1] YU Z, ZHANG Y, JIANG B, FU J, JIN Y. A review on fault-tolerant cooperative control of multiple unmanned aerial vehicles[J]. Chinese Journal of Aeronautics, 2022, 35(1): 1-18. https://doi.org/10.1016/j.cja.2021.04.022.
[2] NGUYEN D-T, SAUSSIE D, SAYDY L. Design and experimental validation of robust self-scheduled fault-tolerant control laws for a multicopter UAV[J]. IEEE/ASME Transactions on Mechatronics, 2021, 26(5): 2548-2557. https://doi.org/10.1109/TMECH.2020.3042333.
[3] COLOMBO L J, GIRIBET J I. Learning-based fault-tolerant control for an hexarotor with model uncertainty[J]. IEEE Transactions on Control Systems Technology, 2024, 32(2): 672-679. https://doi.org/10.1109/TCST.2023.3318855.
[4] YANG K, BAI C, QUAN Q. Multiview target estimation for multicopter swarm interception[J]. IEEE Transactions on Instrumentation and Measurement, 2025, 74: 5023312. https://doi.org/10.1109/TIM.2025.3554914.
[5] PANETSOS F, KARRAS G C, KYRIAKOPOULOS K J. A deep reinforcement learning motion control strategy of a multi-rotor UAV for payload transportation with minimum swing[C]// 2022 30th Mediterranean Conference on Control and Automation (MED). Vouliagmeni, Greece; 2022: 368-374. https://doi.org/10.1109/MED54222.2022.9837220.
[6] AZHAR M A H B, BARTON T E A, ISLAM T. Drone forensic analysis using open source tools[J]. Journal of Digital Forensics, Security and Law, 2018, 13(1). https://doi.org/10.15394/jdfsl.2018.1513.
[7] POSE C D, GIRIBET J I, MAS I. Fault tolerance analysis for a class of reconfigurable aerial hexarotor vehicles[J]. IEEE/ASME Transactions on Mechatronics, 2020, 25(4): 1851-1858. https://doi.org/10.1109/TMECH.2020.2993222.
[8] CHUNG W, SON H. Fault-tolerant control of multirotor UAVs by control variable elimination[J]. IEEE/ASME Transactions on Mechatronics, 2020, 25(5): 2513-2522. https://doi.org/10.1109/TMECH.2020.2982436.
[9] WANG B, ZHANG Y. An adaptive fault-tolerant sliding mode control allocation scheme for multirotor helicopter subject to simultaneous actuator faults[J]. IEEE Transactions on Industrial Electronics, 2018, 65(5): 4227-4236. https://doi.org/10.1109/TIE.2017.2772153.
[10] MAWANZA J T, AGEE J T, BHERO E. Finite-time fault-tolerant time-varying formation control for multiple quadcopter UAVs with communication link faults in wildfire monitoring[C]// 2024 International Conference on Electrical, Computer and Energy Technologies (ICECET). Sydney, Australia; 2024: 1-6. https://doi.org/10.1109/ICECET61485.2024.10698607.
[11] LIU K, WANG R, ZHENG S, DONG S, SUN G. Fixed-time disturbance observer-based robust fault-tolerant tracking control for uncertain quadrotor UAV subject to input delay[J]. Nonlinear Dynamics, 2022, 107(3): 2363-2390. https://doi.org/10.1007/s11071-021-07080-0.
[12] ANAND A. A novel fault tolerant dual-loop control system for autonomous quadcopter navigation[C]// 2023 IEEE MIT Undergraduate Research Technology Conference (URTC). Rancho Palos Verdes, CA, USA; 2023: 1-6. https://doi.org/10.1109/URTC60662.2023.10535024.
[13] HU X, WANG B, SHEN Y, FU Y, LI N. Disturbance observer-enhanced adaptive fault-tolerant control of a quadrotor UAV against actuator faults and disturbances[J]. Drones, 2023, 7: 541. https://doi.org/10.3390/drones7080541.
[14] 郭玉英,廖兰馨,张晓强,等.无人机复合执行器故障有限时间容错控制[J/OL].航空学报,1-14. https://link.cnki.net/urlid/11.1929.V.20251030.1853.014.
[15] ZHOU Y, CHEN Y, ZHANG L, PAN C. Distributed finite-time prescribed performance for multiple unmanned aerial vehicle with time-varying external disturbance[J]. IEEE Internet of Things Journal, 2024, 11(9): 16969-16980. https://doi.org/10.1109/JIOT.2024.3367172.
[16] 魏鹏轩,盛汉霖,李枫韵,等.基于分层自适应的无人机多模式容错控制方法[J/OL].航空学报,1-18. https://link.cnki.net/urlid/11.1929.V.20251215.2049.004.
[17] GAO B, LIU Y-J, LIU L. Adaptive neural fault-tolerant control of a quadrotor UAV via fast terminal sliding mode[J]. Aerospace Science and Technology, 2022, 129: 107818. https://doi.org/10.1016/j.ast.2022.107818.
[18] CUI K, GUAN R, ZHANG W, HE F. A fault-tolerant finite time control method for tilting quadrotors[C]// Proceedings of the 41st Chinese Control Conference. Hefei, China; 2022: 4159-4164. https://doi.org/10.23919/CCC55666.2022.9902359.
[19] YANG S, PAN Y, CAO L, CHEN L. Predefined-time fault-tolerant consensus tracking control for multi-UAV systems with prescribed performance and attitude constraints[J]. IEEE Transactions on Aerospace and Electronic Systems, 2024, 60(4): 4058-4072. https://doi.org/10.1109/TAES.2024.3371406.
[20] ZOU L, GAO C. Adaptive attitude fault-tolerant control of quadcopter based on fuzzy state observer[C]// 2023 International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC). Shenzhen, China; 2023: 154-159. https://doi.org/10.1109/CSIS-IAC60628.2023.10363995.
[21] JUNG W, BANG H. Fault and failure tolerant model predictive control of quadrotor UAV[J]. International Journal of Aeronautical and Space Sciences, 2021, 22: 663-675. https://doi.org/10.1007/s42405-020-00331-1.
[22] KE C, CAI K-Y, QUAN Q. Uniform fault-tolerant control of a quadcopter with rotor failure[J]. IEEE/ASME Transactions on Mechatronics, 2023, 28(1): 507-517. https://doi.org/10.1109/TMECH.2022.3202804.
[23] PAN J, LI Y, CHAI R, XIA S, ZUO L. Multiobjective trajectory planning for UAV-assisted IoT networks based on DRL approach[J]. IEEE Internet of Things Journal, 2025, 12(11): 15840-15852. https://doi.org/10.1109/JIOT.2025.3533584.
[24] RAMEZANI DOORAKI A, LEE D-J. Reinforcement learning based flight controller capable of controlling a quadcopter with four, three and two working motors[C]// 2020 20th International Conference on Control, Automation and Systems (ICCAS). Busan, Korea (South); 2020: 161-166. https://doi.org/10.23919/ICCAS50221.2020.9268270.
[25] DARSIN M, PAMUNGKAS W J, SYUHRI S N H, WIBOWO R K K, BASUKI H A, DJUMHARIANTOD, et al. Advanced optimization of drone frame design through the application of generative design techniques and 3D printing technology[J]. Journal of Mechanical Science and Technology, 2025, 39(1): 119-128. https://doi.org/10.1007/s12206-024-1210-x.
[26] JAYAKUMAR S S, SUBRAMANIAM I P, ARPUTHARAJ B S, SOLAIAAPPAN S K, RAJENDRAN P, LEE I E, et al. Design, control, aerodynamic performances, and structural integrity investigations of compact ducted drone with co?axial propeller for high altitude surveillance[J]. Scientific Reports, 2024, 14(1): 6330. https://doi.org/10.1038/s41598-024-54174-x.
[27] RAJAWAT M. Weather conditions and its effects on UAS[J]. International Research Journal of Modern Engineering and Technology and Science, 2021, 3(12): 255-261.
[28] SHANG X, LI W, YUAN F, ZHI H, GAO Z, GUO M, et al. Research on fault diagnosis of UAV rotor motor bearings based on WPT-CEEMD-CNN-LSTM[J]. Machines, 2025, 13(4): 287. https://doi.org/10.3390/machines13040287.
[29] LEE J-Y, LEE W-T, KO S-H, OH H-S. Fault classification and diagnosis of UAV motor based on estimated nonlinear parameter of steady-state model[J]. International Journal of Mechanical Engineering and Robotics Research, 2021, 10(1): 22-31. https://doi.org/10.1016/j.engappai.2024.109577.
[30] VEINOTT C G. Inherent overheating protection of D-C aircraft motors[J]. Electrical Engineering, 1944, 63(12): 920-924. https://doi.org/10.1109/EE.1944.6440627.
[31] JIA Z, LIU Z, YAO G, WANG K, VONG C. An incremental learning framework for mechanical fault diagnosis with bi-level multiscale convolutional attention[J]. IEEE Transactions on Instrumentation and Measurement, 2025, 74: 3548013. https://doi.org/10.1109/TIM.2025.3581666.
[32] DAI X, QUAN Q, CAI K-Y. Design automation and optimization methodology for electric multicopter unmanned aerial robots[J]. IEEE Transactions on Automation Science and Engineering, 2022, 19(3): 2354-2368. https://doi.org/10.1109/TASE.2021.3081122.
[33] YOO J, JANG D, KIM H J, JOHANSSON K H. Hybrid reinforcement learning control for a micro quadrotor flight[J]. IEEE Control Systems Letters, 2021, 5(2): 505-510. https://doi.org/10.1109/LCSYS.2020.3001663.
[34] SHEHAB H, ZAGHLOUL A, EL-BADAWY A. Low-Level Control of a Quadrotor using Twin Delayed Deep Deterministic Policy Gradient (TD3)[C]// Proceedings of the 2021 18th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE). Mexico City, Mexico; 2021: 1-6. https://doi.org/10.1109/CCE53527.2021.9633086.
[35] LI H, MYSCHIK S, HONG H, HOLZAPFEL F. Rank one update-based efficient adaptive control allocation for multicopter[C]// Proceedings of the 2024 CEAS EuroGNC Conference. Bristol, UK; 2024: 1-16. https://doi.org/10.82124/CEAS-GNC-2024-083.
[36] ERMEYDAN A, KABA A. Feedback linearization based passive fault tolerant control of a quadrotor[C]// 2024 8th International Artificial Intelligence and Data Processing Symposium (IDAP). Malatya, Turkiye; 2024: 1-5. https://doi.org/10.1109/IDAP64064.2024.10710844.

Options

摘要页面

文章导航

模态框（Modal）标题

摘要

本文引用格式

Abstract

参考文献