Render3D: 基于三维渲染的月面立体匹配自监督学习方法
收稿日期: 2025-12-24
修回日期: 2025-12-30
录用日期: 2026-01-05
网络出版日期: 2026-01-15
基金资助
北京市自然科学基金联合重点项目(L258070)
Render3D: A self-supervised learning method for lunar surface stereo matching based on 3D rendering
Received date: 2025-12-24
Revised date: 2025-12-30
Accepted date: 2026-01-05
Online published: 2026-01-15
Supported by
Joint Key Project of Beijing Natural Science Foundation(L258070)
立体视觉以其低成本和高可靠性等优势,成为月面巡视探测中感知三维地形的重要技术路径。近年来,基于深度学习的立体匹配方法已成为实现高精度立体视觉的主流方案。然而,受限于数据采集条件,目前缺乏面向月球表面环境的公开立体匹配数据集,严重制约了深度学习模型在月面场景中的训练与微调,进而影响其对月面复杂地形的适应能力。针对该问题,提出了一种基于三维渲染的立体匹配自监督学习方法,记作Render3D。该方法只需单目月面环拍图像输入,通过融合神经辐射场(NeRF)的精确几何重建能力与2D高斯泼溅(2DGS)的高保真表面渲染能力,生成高质量伪标注训练样本,指导立体匹配模型进行微调以适配月面环境。在月面仿真环境与真实物理场景上的实验表明,经Render3D微调后的立体匹配模型在精度上显著优于其他对比方法。在月面场景下,所提方法明显领先于现有自监督学习方法,尤其在纹理稀疏与高阴影区域等复杂环境下,其匹配误差较基线方法降低约50%,达到最优性能。实验结果充分证实,Render3D方法能够有效缓解月面标注数据稀缺的限制,显著提升立体匹配模型在月面环境中的鲁棒性与泛化能力。
孟偲 , 李依真 , 李俊博 , 梅既澜 , 白相志 , 马江 . Render3D: 基于三维渲染的月面立体匹配自监督学习方法[J]. 航空学报, 2026 , 47(10) : 533278 -533278 . DOI: 10.7527/S1000-6893.2026.33278
Stereo vision has emerged as a key technical approach for perceiving 3D terrain in lunar exploration, owing to its advantages such as low cost and high reliability. In recent years, the stereo matching methods based on deep learning have become the mainstream solution for achieving high-precision stereo vision. However, limited by data acquisition conditions, there is currently a lack of public stereo matching datasets tailored to the lunar surface environment. This severely restricts the training and fine-tuning of deep learning models in lunar surface scenarios, thereby impairing their adaptability to complex lunar terrain. To address this issue, this paper proposes Render3D, a self-supervised learning method for lunar surface stereo matching based on 3D rendering. The proposed method requires only monocular panoramic lunar surface images as input. By integrating the high-fidelity surface rendering capability of 2D Gaussian Splatting (2DGS) with the accurate geometric reconstruction capability of Neural Radiance Fields (NeRF), it generates high-quality pseudo-annotated training samples. These samples guide the fine-tuning of the stereo matching model to adapt to the lunar surface environment. Experiments conducted in both simulated lunar surface environment and real physical scenario demonstrate that the stereo matching model fine-tuned using Render3D method significantly outperforms other methods in terms of accuracy. In lunar surface scenarios, the proposed method shows clear superiority over existing self-supervised learning approaches, particularly in complex conditions such as textureless regions and areas with high shadows, where the matching error is reduced by approximately 50% compared to baseline methods, achieving state-of-the-art performance. Experimental results fully demonstrate that Render3D can effectively alleviate the constraint of scarce labeled data for lunar surface thus significantly improving the robustness and generalization ability of stereo matching models in the lunar surface environment.
| [1] | 王平, 于晓强, 郭继峰. 月球大范围探测巡视器及GNC技术发展综述[J]. 宇航学报, 2022, 43(5): 548-562. |
| WANG P, YU X Q, GUO J F. A survey of lunar wide-range exploration rover and GNC technology[J]. Journal of Astronautics, 2022, 43(5): 548-562 (in Chinese). | |
| [2] | 解杨敏, 季力, 魏祥泉, 等. 国内外行星表面巡视器自主导航技术研究[J]. 上海航天(中英文), 2021, 38(1): 61-71. |
| XIE Y M, JI L, WEI X Q, et al. Domestic and overseas research status on autonomous navigation technology of planetary rovers[J]. Aerospace Shanghai, 2021, 38(1): 61-71 (in Chinese). | |
| [3] | JIA Y T, ZHANG S N, LIU B, et al. A robust method for large-scale route optimization on lunar surface utilizing a multi-level map model[J]. Chinese Journal of Aeronautics, 2025, 38(3): 103388. |
| [4] | LI J B, CHEN K Y, TIAN G J, et al. MarsSeg: Mars surface semantic segmentation with multilevel extractor and connector[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 4501012. |
| [5] | 于晓强, 郭继峰, 赵毓, 等. 月面巡视机器人快速安全路径规划[J]. 航空学报, 2021, 42(1): 524153. |
| YU X Q, GUO J F, ZHAO Y, et al. Fast and safe path planning for lunar rovers[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(1): 524153 (in Chinese). | |
| [6] | JAHANSHAHI H, ZHU Z H. A comprehensive review of tactile sensing technologies in space robotics[J]. Chinese Journal of Aeronautics, 2025, 38(7): 103423. |
| [7] | 陈卓宇, 安丰伟. 面向机器人导航的双目立体视觉处理器综述[J]. 集成电路与嵌入式系统, 2024, 24(11): 15-28. |
| CHEN Z Y, AN F W. Overview of binocular stereo vision processor for robot navigation[J]. Integrated Circuits and Embedded Systems, 2024, 24(11): 15-28 (in Chinese). | |
| [8] | 杨晓立, 徐玉华, 叶乐佳, 等. 双目立体视觉研究进展与应用[J]. 激光与光电子学进展, 2023, 60(8): 0811010. |
| YANG X L, XU Y H, YE L J, et al. Research progress on binocular stereo vision applications[J]. Laser & Optoelectronics Progress, 2023, 60(8): 0811010 (in Chinese). | |
| [9] | 王勇. 基于月面立体导航影像的三维地形重建[D]. 阜新: 辽宁工程技术大学, 2023: 59-61. |
| WANG Y. 3D terrain reconstruction based on lunar stereo navigation images[D]. Fuxin: Liaoning Technical University, 2023: 59-61 (in Chinese). | |
| [10] | ZHANG F H, QI X J, YANG R G, et al. Domain-invariant stereo matching networks[C]∥Computer Vision-ECCV 2020. Cham: Springer, 2020: 420-439. |
| [11] | CAI C J, POGGI M, MATTOCCIA S, et al. Matching-space stereo networks for cross-domain generalization[C]∥2020 International Conference on 3D Vision (3DV). Piscataway: IEEE Press, 2021: 364-373. |
| [12] | ZHANG J W, WANG X, BAI X, et al. Revisiting domain generalized stereo matching networks from a Feature Consistency Perspective[C]∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2022: 12991-13001. |
| [13] | MAYER N, ILG E, H?USSER P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2016: 4040-4048. |
| [14] | GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: The KITTI dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237. |
| [15] | 刘传凯, 王沼翔, 雷俊雄, 等. 基于松弛极线约束的月面复杂仿射变换图像匹配方法[J]. 航空学报, 2024, 45(2): 328659. |
| LIU C K, WANG Z X, LEI J X, et al. An epipolar relaxation constrained matching algorithm of large-affined images for lunar rover with large span distance in a single movement[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(2): 328659 (in Chinese). | |
| [16] | GARG R, VIJAY KUMAR B G, CARNEIRO G, et al. Unsupervised CNN for single view depth estimation: geometry to the rescue[M]∥Computer Vision-ECCV 2016. Cham: Springer, 2016: 740-756. |
| [17] | REN Z, YAN J C, NI B B, et al. Unsupervised deep learning for optical flow estimation[C]∥Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2017, 1495-1501. |
| [18] | GODARD C, AODHA O MAC, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2017: 6602-6611. |
| [19] | ZHANG Y M, CHEN Y M, BAI X, et al. Adaptive unimodal cost volume filtering for deep stereo matching[C]∥Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020, 34(7): 12926-12934. |
| [20] | POGGI M, TONIONI A, TOSI F, et al. Continual adaptation for deep stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 4713-4729. |
| [21] | TONIONI A, POGGI M, MATTOCCIA S, et al. Unsupervised adaptation for deep stereo[C]∥2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE Press, 2017: 1614-1622. |
| [22] | TONIONI A, POGGI M, MATTOCCIA S, et al. Unsupervised domain adaptation for depth prediction from images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(10): 2396-2409. |
| [23] | WATSON J, AODHA O MAC, TURMUKHAMBETOV D, et al. Learning stereo from single images[C]∥Computer Vision-ECCV 2020. Cham: Springer, 2020: 722-740. |
| [24] | ALEOTTI F, TOSI F, ZHANG L, et al. Reversing the cycle: self-supervised deep stereo through enhanced monocular distillation[C]∥Computer Vision-ECCV 2020. Cham: Springer, 2020: 614-632. |
| [25] | TOSI F, TONIONI A, DE GREGORIO D, et al. NeRF-supervised deep stereo[C]∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2023: 855-866. |
| [26] | MARTIN-BRUALLA R, RADWAN N, SAJJADI M S M, et al. NeRF in the wild: Neural radiance fields for unconstrained photo collections[C]∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2021: 7206-7215. |
| [27] | HUANG L T, BAI J Y, GUO J, et al. On the error analysis of 3D Gaussian splatting and an optimal projection strategy[C]∥Computer Vision-ECCV 2024. Cham: Springer, 2024: 247-263. |
| [28] | MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: Representing scenes as neural radiance fields for view synthesis[J]. 2021, 65(1): 99-106. |
| [29] | HUANG B B, YU Z H, CHEN A P, et al. 2D Gaussian splatting for geometrically accurate radiance fields[C]∥ACM SIGGRAPH 2024 Conference Papers. New York: ACM, 2024: 1-11. |
| [30] | KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-end learning of geometry and context for deep stereo regression[C]∥2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE Press, 2017: 66-75. |
| [31] | CHANG J R, CHEN Y S. Pyramid stereo matching network[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 5410-5418. |
| [32] | ZHANG F H, PRISACARIU V, YANG R G, et al. GA-net: guided aggregation net for end-to-end stereo matching[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2020: 185-194. |
| [33] | GUO X Y, YANG K, YANG W K, et al. Group-wise correlation stereo network[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2020: 3268-3277. |
| [34] | SHEN Z L, DAI Y C, RAO Z B. CFNet: cascade and fused cost volume for robust stereo matching[C]∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2021: 13901-13910. |
| [35] | GU X D, FAN Z W, DAI Z Z, et al. Cascade cost volume for high-resolution multi-view stereo and Stereo Matching[C]∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2020: 2495-2504. |
| [36] | LI Z, LIU X, DRENKOW N, et al. Revisiting stereo depth estimation from a sequence-to-sequence perspective with transformers[C]∥2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2021: 6197-6206. |
| [37] | XU G, CHENG J, GUO P, et al. Attention concatenation volume for accurate and efficient stereo matchin g[C]∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2022: 12981-12990. |
| [38] | 孔令寅, 朱江平, 应三丛. 基于引导图像和自适应支持域的立体匹配[J]. 光学学报, 2020, 40(9): 0915001. |
| KONG L Y, ZHU J P, YING S C. Stereo matching based on guidance image and adaptive support region[J]. Acta Optica Sinica, 2020, 40(9): 0915001 (in Chinese). | |
| [39] | LIPSON L, TEED Z, DENG J. RAFT-stereo: multilevel recurrent field transforms for stereo matching[C]∥2021 International Conference on 3D Vision (3DV). Piscataway: IEEE Press, 2022: 218-227. |
| [40] | XU G W, WANG X Q, DING X H, et al. Iterative geometry encoding volume for stereo matching[C]∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2023: 21919-21928. |
| [41] | XU G W, WANG X Q, ZHANG Z X, et al. IGEV++: Iterative multi-range geometry encoding volumes for stereo matching[EB/OL]. (2025-05-11) [2025-10-01]: . |
| [42] | SCH?NBERGER J L, FRAHM J M. Structure-from-motion revisited[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2016: 4104-4113. |
| [43] | SHAH S, DEY D, LOVETT C, et al. AirSim: High-fidelity visual and physical simulation for autonomous vehicles[C]∥Field and Service Robotics. Cham: Springer, 2018: 621-635. |
| [44] | ROMáN R, GONZáLEZ R, TOLEDANO C, et al. Correction of a lunar-irradiance model for aerosol optical depth retrieval and comparison with a star photometer[J]. Atmospheric Measurement Techniques, 2020, 13(11): 6293-6310. |
| [45] | STUBBS T J, VONDRAK R R, FARRELL W M. A dynamic fountain model for lunar dust[J]. Advances in Space Research, 2006, 37(1): 59-66. |
| [46] | HOLLINGSWORTH D K, WITTE L C, HINKE J, et al. Reduction in emittance of thermal radiator coatings caused by the accumulation of a Martian dust simulant[J]. Applied Thermal Engineering, 2006, 26(17-18): 2383-2392. |
| [47] | LIU J Y, ZHANG Q Y, WAN X, et al. LuSNAR: A lunar segmentation, navigation and reconstruction dataset based on Muti-sensor for autonomous exploration[EB/OL]. (2024-09-26)[2025-10-01]: . |
| [48] | HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 328-341. |
| [49] | MARTULL S, PERIS M, FUKUI K. Realistic CG stereo image dataset with ground truth disparity maps: IEICE PRMU-430[R]. Tokyo: IEICE, 2012. |
| [50] | MEI X, SUN X, ZHOU M C, et al. On building an accurate stereo matching system on graphics hardware[C]∥2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). Piscataway: IEEE Press, 2012: 467-474. |
| [51] | SCH?PS T, SCH?NBERGER J L, GALLIANI S, et al. A multi-view stereo benchmark with high-resolution images and multi-camera videos[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2017: 2538-2547. |
/
| 〈 |
|
〉 |