基于残差混合监督网络的无人机目标阴影检测
收稿日期: 2024-01-02
修回日期: 2024-01-19
录用日期: 2024-03-15
网络出版日期: 2024-03-29
基金资助
国家自然科学基金(52072309);陕西省重点研发计划(2019ZDLGY14-02-01);深圳市基础研究资助项目(JCYJ20190806152203506);航空科学基金(ASFC-2018ZC53026)
Shadow detection of UAV target based on residual mixed supervision network
Received date: 2024-01-02
Revised date: 2024-01-19
Accepted date: 2024-03-15
Online published: 2024-03-29
Supported by
National Natural Science Foundation of China(52072309);Key Research and Development Program of Shaanxi
无人机在检测和跟踪目标过程中受到目标伪装、目标遮挡、移动躲避以及假目标等因素的干扰，而无人机目标附近的阴影区域加剧了这些因素对目标检测和跟踪性能的影响，因此检测无人机目标阴影区域是无人机领域的重要研究任务之一。现有无人机目标阴影检测方法面临训练数据数量有限、数据收集标注困难以及无人机目标中存在大量尺寸较小的细碎阴影区域等问题，针对这些问题，提出一种基于残差混合监督网络的无人机目标阴影检测算法。首先针对无人机目标阴影检测任务的特点设计分辨率注意力网络，在结合底层纹理特征和高层语义特征的过程中，更准确地保留底层纹理特征。然后设计混合监督网络扩充训练数据集，结合普通阴影检测数据集和无人机目标阴影检测数据集训练教师网络，使用无人机阴影检测数据集和教师网络的参数训练学生网络。同时设计残差图像，利用教师网络检测结果和标准结果之间的残差图像扩充训练数据集，使阴影检测网络更加关注细碎阴影区域。最后，在2个公开实验数据集上和已有方法进行对比实验，在各个评价参数上取得了最多41.6%的提升效果，证明所提无人机目标阴影检测算法较好的解决了现有方法存在的问题，具有较高的准确性。
王潇 , 刘贞报 , 史忠科 . 基于残差混合监督网络的无人机目标阴影检测[J]. 航空学报, 2024 , 45(17) : 530062 -530062 . DOI: 10.7527/S1000-6893.2024.30062
The targets camouflage， targets occlusion， moving dodge and fake targets deteriorate the performance of UAV object detection and tracking， and the shadow regions around UAV target aggravate the negative effect of these factors. Thus， shadow detection is an important task for UAV. The shadow detection of UAV target suffering from limited training images， difficulty in labeling ground truth data and mass of tiny shadow regions. To deal with these problems， we propose a UAV target shadow detection method based on residual mixed supervision network. Firstly， we design a resolution-aware attention shadow detection network based on the character of shadow regions in UAV target. The newly designed network can maintain the lower texture feature more accurately. Then we design mixed supervision network to enlarge the number of training images. The teacher network is trained by both ordinary dataset and UAV dataset， while the student network is trained based on UAV dataset and the parameter of teacher network. Meanwhile we design residual images to further enlarge the number of training images and makes the network pay more attention to tiny shadow regions. The residual image is calculated by measuring the difference between detection results of teacher network and ground truth data. At last， the proposed method is compared with existing methods on two public UAV target shadow detection datasets. The evaluation metrics are improved by 41.6% at most. The experiment proves the effectiveness and accuracy of proposed shadow detection method on UAV target.
| 1 | 石争浩， 仵晨伟， 李成建， 等. 航空遥感图像深度学习目标检测技术研究进展［J］. 中国图象图形学报， 2023， 28（9）： 2616-2643. |
| SHI Z H， WU C W， LI C J， et al. Object detection techniques based on deep learning for aerial remote sensing images： A survey［J］. Journal of Image and Graphics， 2023， 28（9）： 2616-2643 （in Chinese）. | |
| 2 | 刘贞报， 马博迪， 高红岗， 等. 基于形态自适应网络的无人机目标跟踪方法［J］. 航空学报， 2021， 42（4）： 524904. |
| LIU Z B， MA B D， GAO H G， et al. Adaptive morphological network based UAV target tracking algorithm［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（4）： 524904 （in Chinese）. | |
| 3 | 江波， 屈若锟， 李彦冬， 等. 基于深度学习的无人机航拍目标检测研究综述［J］. 航空学报， 2021， 42（4）： 524519. |
| JIANG B， QU R K， LI Y D， et al. Object detection in UAV imagery based on deep learning： Review［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（4）： 524519 （in Chinese）. | |
| 4 | 刘芳， 韩笑. 基于多尺度深度学习的自适应航拍目标检测［J］. 航空学报， 2022， 43（5）： 325270. |
| LIU F， HAN X. Adaptive aerial object detection based on multi-scale deep learning［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（5）： 325270 （in Chinese）. | |
| 5 | 施文灶， 毛政元. 基于图割与阴影邻接关系的高分辨率遥感影像建筑物提取方法［J］. 电子学报， 2016， 44（12）： 2849-2854. |
| SHI W Z， MAO Z Y. Building extraction from high resolution remotely sensed imagery based on shadows and graph-cut segmentation［J］. Acta Electronica Sinica， 2016， 44（12）： 2849-2854 （in Chinese）. | |
| 6 | LI Y， WANG H N， FANG Y Q， et al. Learning power Gaussian modeling loss for dense rotated object detection in remote sensing images［J］. Chinese Journal of Aeronautics， 2023， 36（10）： 353-365. |
| 7 | GUO R Q， DAI Q Y， HOIEM D. Paired regions for shadow detection and removal［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2013， 35（12）： 2956-2967. |
| 8 | LUO S， LI H F， SHEN H F. Deeply supervised convolutional neural network for shadow detection based on a novel aerial shadow imagery dataset［J］. ISPRS Journal of Photogrammetry and Remote Sensing， 2020， 167： 443-457. |
| 9 | LUO S， LI H F， ZHU R Z， et al. ESPFNet： An edge-aware spatial pyramid fusion network for salient shadow detection in aerial remote sensing images［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2021， 14： 4633-4646. |
| 10 | LIU D Y， ZHANG J P， WU Y H， et al. A shadow detection algorithm based on multiscale spatial attention mechanism for aerial remote sensing images［J］. IEEE Geoscience and Remote Sensing Letters， 2022， 19： 6003905. |
| 11 | ZHU Q Q， YANG Y， SUN X L， et al. CDANet： contextual detail-aware network for high-spatial-resolution remote-sensing imagery shadow detection［J］. IEEE Transactions on Geoscience and Remote Sensing， 2022， 60： 5617415. |
| 12 | ZHANG J， SHI X L， ZHENG C Y， et al. MRPFA-net for shadow detection in remote-sensing images［J］. IEEE Transactions on Geoscience and Remote Sensing， 2023， 61： 5514011. |
| 13 | 林雨准， 张保明， 郭海涛， 等. 结合多尺度分割和形态学运算的高分辨率遥感影像阴影检测［J］. 中国图象图形学报， 2018， 23（8）： 1263-1272. |
| LIN Y Z， ZHANG B M， GUO H T， et al. Shadow detection from high resolution remote sensing imagery based on multi-scale segmentation and morphology operation［J］. Journal of Image and Graphics， 2018， 23（8）： 1263-1272 （in Chinese）. | |
| 14 | VICENTE T F Y， HOAI M， SAMARAS D. Leave-one-out kernel optimization for shadow detection and removal［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018， 40（3）： 682-695. |
| 15 | KHAN S H， BENNAMOUN M， SOHEL F， et al. Automatic shadow detection and removal from a single image［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38（3）： 431-446. |
| 16 | VICENTE T F Y， HOU L， YU C P， et al. Large-scale training of shadow detectors with noisily-annotated shadow examples［C］∥European Conference on Computer Vision. Cham： Springer， 2016： 816-832. |
| 17 | NGUYEN V， VICENTE T F Y， ZHAO M Z， et al. Shadow detection with conditional generative adversarial networks［C］∥ 2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 4520-4528. |
| 18 | LE H， VICENTE T F Y， NGUYEN V， et al. A+D net： Training a shadow detector with adversarial shadow attenuation［C］∥European Conference on Computer Vision. Cham： Springer， 2018： 680-696. |
| 19 | WANG J F， LI X， YANG J. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal［C］∥ 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 1788-1797. |
| 20 | HU X W， FU C W， ZHU L， et al. Direction-aware spatial context features for shadow detection and removal［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（11）： 2795-2808. |
| 21 | ZHU L， DENG Z J， HU X W， et al. Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection［C］∥European Conference on Computer Vision. Cham： Springer， 2018： 122-137. |
| 22 | ZHENG Q L， QIAO X T， CAO Y， et al. Distraction-aware shadow detection［C］∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2019： 5162-5171. |
| 23 | CHEN Z H， ZHU L， WAN L， et al. A multi-task mean teacher for semi-supervised shadow detection［C］∥ 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 5610-5619. |
| 24 | JIN Y W， XU W B， HU Z W， et al. GSCA-UNet： towards automatic shadow detection in urban aerial imagery with global-spatial-context attention module［J］. Remote Sensing， 2020， 12（17）： 2864. |
| 25 | TARVAINEN A， VALPOLA H. Mean teachers are better role models： Weight-averaged consistency targets improve semi-supervised deep learning results［C］∥ Proceedings of the 31st International Conference on Neural Information Processing Systems. New York：ACM， 2017：1195–1204. |
| 26 | LIU Z， MAO H Z， WU C Y， et al. A ConvNet for the 2020s［C］∥ 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 11966-11976. |
| 27 | LIU Z， LIN Y T， CAO Y， et al. Swin Transformer： Hierarchical vision transformer using shifted windows［C］∥ 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2021： 9992-10002. |
| 28 | DONG X Y， BAO J M， CHEN D D， et al. CSWin transformer： A general vision transformer backbone with cross-shaped windows［C］∥ 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 12114-12124. |
| 29 | ZHANG Y D， CHEN G， VUKOMANOVIC J， et al. Recurrent Shadow Attention Model （RSAM） for shadow removal in high-resolution urban land-cover mapping［J］. Remote Sensing of Environment， 2020， 247： 111945. |
| 30 | HE Z J， ZHANG Z Z， GUO M Q， et al. Adaptive unsupervised-shadow-detection approach for remote-sensing image based on multichannel features［J］. Remote Sensing， 2022， 14（12）： 2756. |
| 31 | YANG Y， GUO M Q， ZHU Q Q. CADNet： Top-down contextual saliency detection network for high spatial resolution remote sensing image shadow detection［C］∥ 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. Piscataway： IEEE Press， 2021： 4075-4078. |
| 32 | HU X W， WANG T Y， FU C W， et al. Revisiting shadow detection： A new benchmark dataset for complex world［J］. IEEE Transactions on Image Processing， 2021， 30： 1925-1934. |
| 33 | RONNEBERGER O， FISCHER P， BROX T. U-net： Convolutional networks for biomedical image segmentation［C］∥International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham： Springer， 2015： 234-241. |
| 34 | ZHU J J， SAMUEL K G G， MASOOD S Z， et al. Learning to recognize shadows in monochromatic natural images［C］∥ 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2010： 223-230. |
/
| 〈 |
|
〉 |
CJA