航空学报 > 2023, Vol. 44 Issue (18): 28334-028334   doi: 10.7527/S1000-6893.2023.28334

基于深度学习的无人机航拍视频多目标检测与跟踪研究进展

苑玉彬, 吴一全(), 赵朗月, 陈金林, 赵其昌   

  1. 南京航空航天大学 电子信息工程学院,南京 211106
  • 收稿日期:2022-11-29 修回日期:2022-12-31 接受日期:2023-03-16 出版日期:2023-09-25 发布日期:2023-03-21
  • 通讯作者: 吴一全 E-mail:nuaaimage@163.com
  • 基金资助:
    国家自然科学基金(61573183)

Research progress of UAV aerial video multi⁃object detection and tracking based on deep learning

Yubin YUAN, Yiquan WU(), Langyue ZHAO, Jinlin CHEN, Qichang ZHAO   

  1. College of Electronic and Information Engineering,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China
  • Received:2022-11-29 Revised:2022-12-31 Accepted:2023-03-16 Online:2023-09-25 Published:2023-03-21
  • Contact: Yiquan WU E-mail:nuaaimage@163.com
  • Supported by:
    National Natural Science Foundation of China(61573183)

摘要:

随着无人机航拍的数据采集愈加便捷,以无人机为平台的多目标检测与跟踪技术发展迅速,在智慧城市、环境监测、地质探测、精准农业和灾害预警等民用和军事领域有着广泛的应用前景,近年来深度学习的突飞猛进也为其提供了多种更为有效的解决思路。然而,无人机视角下目标外观发生突变、目标区域被严重遮挡以及目标消失和重现等挑战性的问题尚未完全解决。综述了基于深度学习的无人机航拍视频多目标检测与跟踪算法,总结了该领域的最新进展,包括多目标检测、多目标跟踪2个模块。多目标检测模块划分为双阶段与单阶段两个部分。对于多目标跟踪模块则依据基于检测的跟踪和联合检测的跟踪2个经典框架,分别阐述了2类算法的原理并分析其优缺点。随后对现有的公开数据集进行统计分析,并对基于无人机航拍视频的多目标检测与跟踪领域内标杆挑战赛VisDrone Challenge近年来的最优方案进行了对比分析。最后总结了无人机视角下多目标检测与跟踪亟待解决的问题并展望未来可能的研究方向,为后续相关研究的人员提供参考。

关键词: 无人机航拍视频, 多目标检测, 多目标跟踪, 深度学习, 单阶段检测, 双阶段检测, 检测跟踪, 联合检测跟踪

Abstract:

With the increasing convenience of data acquisition for aerial photography of Unmanned Aerial Vehicle (UAV), the multi-target detection and tracking technology based on the UAV platform has developed rapidly and has broad prospects for applications in civil and military fields. In recent years, the rapid progress of in-depth learning has also provided a variety of more effective solutions. However, the challenging problems such as sudden changes in the appearance of the target, serious occlusion of the target area, and disappearance and reappearance of the target from the perspective of UAV have not been completely solved. In this paper, we summarize the algorithms for multi-target detection and tracking in UAV aerial video based on deep learning, and summarize the latest progress in this field, including multi-target detection and multi-object tracking. The multi-object detection module is divided into two parts: two-stage and one-stage detection. For the multi-object tracking module, according to the two classical frameworks of tracking-based detection and joint-detection tracking, the principles of the two algorithms are described and their advantages and disadvantages are analyzed. Then, the existing public data sets are statistically analyzed, and the optimal schemes of the benchmark challenge VisDrone Challenge in the field of multi-target detection and tracking based on UAV aerial video in recent years are compared and analyzed. Finally, the paper discusses the urgent problems of multi-object detection and tracking from the perspective of UAV and the possible research directions in the future, providing a reference for the follow-up researchers.

Key words: UAV aerial video, multi object detection, multi target tracking, deep learning, one-stage detection, two-stage detection, detection tracking, joint-detection tracking

中图分类号: