ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Generalized few-shot segmentation for remote sensing image based on class relation mining
Received date: 2024-12-19
Revised date: 2025-03-04
Accepted date: 2025-04-23
Online published: 2025-05-06
Supported by
Taishan Scholar Project of Shandong Province(tsqn202306079);National Natural Science Foundation of China(62471278)
With the rapid development of remote sensing satellites and unmanned aerial vehicle technologies， the acquisition of remote sensing images has become more convenient， making precise interpretation of image content a research hotspot. However， existing remote sensing image semantic segmentation methods struggle to handle new object categories that emerge due to scene changes when only a small number of samples are available， thereby affecting the overall performance of the model. To address the potential issues of new classes and new scenes in remote sensing scenarios， this paper introduces the generalized few-shot segmentation technique， aiming to rapidly adapt to new classes using a few samples and efficiently complete the semantic segmentation task for remote sensing images. Firstly， to tackle the challenge of distinguishing objects of different categories and different sizes in remote sensing images， a multi-scale feature fusion decoder is introduced to enhance the model’s segmentation ability for targets of different scales. Then， to address the background semantic shift problem in the generalized few-shot segmentation task， a background consistency modeling training strategy is designed to ensure the accurate expression of background features by the model. Furthermore， this paper proposes an inter-class relationship mining technique， which utilizes prototypes generated from a large amount of base class data to enhance the prototypes generated from a small amount of new class data， enabling them to acquire beneficial information from the base classes and achieve more precise segmentation. Finally， comparative experiments with mainstream methods are conducted on public datasets， and the mIoU segmentation metric is improved by up to 11.48%， demonstrating the effectiveness of the proposed algorithm.
Runmin CONG , Haoyan SUN , Yuxuan LUO , Hao FANG . Generalized few-shot segmentation for remote sensing image based on class relation mining[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2025 , 46(23) : 631694 -631694 . DOI: 10.7527/S1000-6893.2025.31694
| [1] | DIAKOGIANNIS F I， WALDNER F， CACCETTA P， et al. ResUNet-a： A deep learning framework for semantic segmentation of remotely sensed data［J］. ISPRS Journal of Photogrammetry and Remote Sensing， 2020， 162： 94-114. |
| [2] | 罗旭东， 吴一全， 陈金林. 无人机航拍影像目标检测与语义分割的深度学习方法研究进展［J］. 航空学报， 2024， 45（6）： 028822. |
| LUO X D， WU Y Q， CHEN J L. Research progress on deep learning methods for object detection and semantic segmentation in UAV aerial images［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（6）： 028822 （in Chinese）. | |
| [3] | 张文凯， 刘文杰， 孙显， 等. 多源特征自适应融合网络的高分遥感影像语义分割［J］. 中国图象图形学报， 2022， 27（8）： 2516-2526. |
| ZHANG W K， LIU W J， SUN X， et al. Multi-source features adaptation fusion network for semantic segmentation in high-resolution remote sensing images［J］. Journal of Image and Graphics， 2022， 27（8）： 2516-2526 （in Chinese）. | |
| [4] | 杨俊俐， 姜志国， 周全， 等. 基于条件随机场的遥感图像语义标注［J］. 航空学报， 2015， 36（9）： 3069-3081. |
| YANG J L， JIANG Z G， ZHOU Q， et al. Remote sensing image semantic labeling based on conditional random field［J］. Acta Aeronautica et Astronautica Sinica， 2015， 36（9）： 3069-3081 （in Chinese）. | |
| [5] | HUA Y S， MARCOS D， MOU L C， et al. Semantic segmentation of remote sensing images with sparse annotations［J］. IEEE Geoscience and Remote Sensing Letters， 2021， 19： 8006305. |
| [6] | SONG N， ZHANG C， LIN G S. Few-shot open-set recognition using background as unknowns［C］∥Proceedings of the 30th ACM International Conference on Multimedia. New York： ACM， 2022： 5970-5979. |
| [7] | ZHANG Y， YANG J W， TIAN J， et al. Modality-aware mutual learning for multi-modal medical image segmentation［M］∥Medical Image Computing and Computer Assisted Intervention-MICCAI 2021. Cham： Springer International Publishing， 2021： 589-599. |
| [8] | 文贡坚， 王润生. 从航空遥感图像中自动提取主要道路［J］. 软件学报， 2000， 11（7）： 957-964. |
| WEN G J， WANG R S. Automatic extraction of main roads from aerial remote sensing images［J］. Journal of Software， 2000， 11（7）： 957-964 （in Chinese）. | |
| [9] | SPASEV V， DIMITROVSKI I， KITANOVSKI I， et al. Semantic segmentation of remote sensing images： definition， methods， datasets and applications［M］∥ICT Innovations 2023. Learning： Humans， Theory， Machines， and Data. Cham： Springer Nature Switzerland， 2024： 127-140. |
| [10] | WANG K X， LIEW J H， ZOU Y T， et al. PANet： Few-shot image semantic segmentation with prototype alignment［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2019： 9196-9205. |
| [11] | FAN Q， PEI W J， TAI Y W， et al. Self-support few-shot semantic segmentation［M］∥Computer Vision-ECCV 2022. Cham： Springer Nature Switzerland， 2022： 701-719. |
| [12] | PENG B H， TIAN Z T， WU X Y， et al. Hierarchical dense correlation distillation for few-shot segmentation［C］∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 23641-23651. |
| [13] | WANG J， LI J Y， CHEN C， et al. Adaptive FSS： a novel few-shot segmentation framework via prototype enhancement［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2024， 38（6）： 5463-5471. |
| [14] | VILALTA R， DRISSI Y. A perspective view and survey of meta-learning［J］. Artificial Intelligence Review， 2002， 18（2）： 77-95. |
| [15] | ZHUANG F Z， QI Z Y， DUAN K Y， et al. A comprehensive survey on transfer learning［J］. Proceedings of the IEEE， 2021， 109（1）： 43-76. |
| [16] | KANG D， CHO M. Integrative few-shot learning for classification and segmentation［C］∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 9969-9980. |
| [17] | TIAN Z T， LAI X， JIANG L， et al. Generalized few-shot semantic segmentation［C］∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 11553-11562. |
| [18] | HAJIMIRI S， BOUDIAF M， AYED I BEN， et al. A strong baseline for generalized few-shot semantic segmentation［C］∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 11269-11278. |
| [19] | LIU S N， ZHANG Y H， QIU Z F， et al. Learning orthogonal prototypes for generalized few-shot semantic segmentation［C］∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 11319-11328. |
| [20] | LIU W D， WU Z H， ZHAO Y， et al. Harmonizing base and novel classes： A class-contrastive approach for generalized few-shot segmentation［J］. International Journal of Computer Vision， 2024， 132（4）： 1277-1291. |
| [21] | BRONI-BEDIAKO C， XIA J S， SONG J， et al. Generalized few-shot semantic segmentation in remote sensing： Challenge and benchmark［J］. IEEE Geoscience and Remote Sensing Letters， 2024， 21： 8003905. |
| [22] | LI Z H， LU F X， ZOU J Q， et al. Generalized few-shot meets remote sensing： Discovering novel classes in land cover mapping via hybrid semantic segmentation framework［C］∥2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Piscataway： IEEE Press， 2024： 2744-2754. |
| [23] | GAO T Y， AO W， WANG X G， et al. Enrich， distill and fuse： Generalized few-shot semantic segmentation in remote sensing leveraging foundation model’s assistance［C］∥2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Piscataway： IEEE Press， 2024： 2771-2780. |
| [24] | ZHU J G， TANG S X， CHEN D P， et al. Complementary relation contrastive distillation［C］∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 9256-9265. |
| [25] | 赵翔， 李洪双. 基于交叉熵和空间分割的全局可靠性灵敏度分析［J］. 航空学报， 2018， 39（2）： 221570. |
| ZHAO X， LI H S. Global reliability sensitivity analysis using cross entropy method and space partition［J］. Acta Aeronautica et Astronautica Sinica， 2018， 39（2）： 221570 （in Chinese）. | |
| [26] | XIA J S， YOKOYA N， ADRIANO B， et al. Open-EarthMap： A benchmark dataset for global high-resolution land cover mapping［C］∥2023 IEEE/CVF Winter Conference on Applications of Computer Vision （WACV）. Piscataway： IEEE Press， 2023： 6243-6253. |
| [27] | SYED W Z， ARORA A， GUPTA A， et al. iSAID： A larg-e-scale dataset for instance segmentation in aerial images［DB/OL］.arXiv preprint， 1905.12886， 2019. |
| [28] | HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］∥2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 770-778. |
| [29] | LIU Z， LIN Y T， CAO Y， et al. Swin transformer： hierarchical vision transformer using shifted windows［C］∥2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2021： 9992-10002. |
/
| 〈 |
|
〉 |
CJA