ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Render3D: A self-supervised learning method for lunar surface stereo matching based on 3D rendering
Received date: 2025-12-24
Revised date: 2025-12-30
Accepted date: 2026-01-05
Online published: 2026-01-15
Supported by
Joint Key Project of Beijing Natural Science Foundation(L258070)
Stereo vision has emerged as a key technical approach for perceiving 3D terrain in lunar exploration， owing to its advantages such as low cost and high reliability. In recent years， the stereo matching methods based on deep learning have become the mainstream solution for achieving high-precision stereo vision. However， limited by data acquisition conditions， there is currently a lack of public stereo matching datasets tailored to the lunar surface environment. This severely restricts the training and fine-tuning of deep learning models in lunar surface scenarios， thereby impairing their adaptability to complex lunar terrain. To address this issue， this paper proposes Render3D， a self-supervised learning method for lunar surface stereo matching based on 3D rendering. The proposed method requires only monocular panoramic lunar surface images as input. By integrating the high-fidelity surface rendering capability of 2D Gaussian Splatting （2DGS） with the accurate geometric reconstruction capability of Neural Radiance Fields （NeRF）， it generates high-quality pseudo-annotated training samples. These samples guide the fine-tuning of the stereo matching model to adapt to the lunar surface environment. Experiments conducted in both simulated lunar surface environment and real physical scenario demonstrate that the stereo matching model fine-tuned using Render3D method significantly outperforms other methods in terms of accuracy. In lunar surface scenarios， the proposed method shows clear superiority over existing self-supervised learning approaches， particularly in complex conditions such as textureless regions and areas with high shadows， where the matching error is reduced by approximately 50% compared to baseline methods， achieving state-of-the-art performance. Experimental results fully demonstrate that Render3D can effectively alleviate the constraint of scarce labeled data for lunar surface thus significantly improving the robustness and generalization ability of stereo matching models in the lunar surface environment.
Cai MENG , Yizhen LI , Junbo LI , Jilan MEI , Xiangzhi BAI , Jiang MA . Render3D: A self-supervised learning method for lunar surface stereo matching based on 3D rendering[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2026 , 47(10) : 533278 -533278 . DOI: 10.7527/S1000-6893.2026.33278
| [1] | 王平， 于晓强， 郭继峰. 月球大范围探测巡视器及GNC技术发展综述［J］. 宇航学报， 2022， 43（5）： 548-562. |
| WANG P， YU X Q， GUO J F. A survey of lunar wide-range exploration rover and GNC technology［J］. Journal of Astronautics， 2022， 43（5）： 548-562 （in Chinese）. | |
| [2] | 解杨敏， 季力， 魏祥泉， 等. 国内外行星表面巡视器自主导航技术研究［J］. 上海航天（中英文）， 2021， 38（1）： 61-71. |
| XIE Y M， JI L， WEI X Q， et al. Domestic and overseas research status on autonomous navigation technology of planetary rovers［J］. Aerospace Shanghai， 2021， 38（1）： 61-71 （in Chinese）. | |
| [3] | JIA Y T， ZHANG S N， LIU B， et al. A robust method for large-scale route optimization on lunar surface utilizing a multi-level map model［J］. Chinese Journal of Aeronautics， 2025， 38（3）： 103388. |
| [4] | LI J B， CHEN K Y， TIAN G J， et al. MarsSeg： Mars surface semantic segmentation with multilevel extractor and connector［J］. IEEE Transactions on Geoscience and Remote Sensing， 2025， 63： 4501012. |
| [5] | 于晓强， 郭继峰， 赵毓， 等. 月面巡视机器人快速安全路径规划［J］. 航空学报， 2021， 42（1）： 524153. |
| YU X Q， GUO J F， ZHAO Y， et al. Fast and safe path planning for lunar rovers［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（1）： 524153 （in Chinese）. | |
| [6] | JAHANSHAHI H， ZHU Z H. A comprehensive review of tactile sensing technologies in space robotics［J］. Chinese Journal of Aeronautics， 2025， 38（7）： 103423. |
| [7] | 陈卓宇， 安丰伟. 面向机器人导航的双目立体视觉处理器综述［J］. 集成电路与嵌入式系统， 2024， 24（11）： 15-28. |
| CHEN Z Y， AN F W. Overview of binocular stereo vision processor for robot navigation［J］. Integrated Circuits and Embedded Systems， 2024， 24（11）： 15-28 （in Chinese）. | |
| [8] | 杨晓立， 徐玉华， 叶乐佳， 等. 双目立体视觉研究进展与应用［J］. 激光与光电子学进展， 2023， 60（8）： 0811010. |
| YANG X L， XU Y H， YE L J， et al. Research progress on binocular stereo vision applications［J］. Laser & Optoelectronics Progress， 2023， 60（8）： 0811010 （in Chinese）. | |
| [9] | 王勇. 基于月面立体导航影像的三维地形重建［D］. 阜新： 辽宁工程技术大学， 2023： 59-61. |
| WANG Y. 3D terrain reconstruction based on lunar stereo navigation images［D］. Fuxin： Liaoning Technical University， 2023： 59-61 （in Chinese）. | |
| [10] | ZHANG F H， QI X J， YANG R G， et al. Domain-invariant stereo matching networks［C］∥Computer Vision-ECCV 2020. Cham： Springer， 2020： 420-439. |
| [11] | CAI C J， POGGI M， MATTOCCIA S， et al. Matching-space stereo networks for cross-domain generalization［C］∥2020 International Conference on 3D Vision （3DV）. Piscataway： IEEE Press， 2021： 364-373. |
| [12] | ZHANG J W， WANG X， BAI X， et al. Revisiting domain generalized stereo matching networks from a Feature Consistency Perspective［C］∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 12991-13001. |
| [13] | MAYER N， ILG E， H?USSER P， et al. A large dataset to train convolutional networks for disparity， optical flow， and scene flow estimation［C］∥2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 4040-4048. |
| [14] | GEIGER A， LENZ P， STILLER C， et al. Vision meets robotics： The KITTI dataset［J］. The International Journal of Robotics Research， 2013， 32（11）： 1231-1237. |
| [15] | 刘传凯， 王沼翔， 雷俊雄， 等. 基于松弛极线约束的月面复杂仿射变换图像匹配方法［J］. 航空学报， 2024， 45（2）： 328659. |
| LIU C K， WANG Z X， LEI J X， et al. An epipolar relaxation constrained matching algorithm of large-affined images for lunar rover with large span distance in a single movement［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（2）： 328659 （in Chinese）. | |
| [16] | GARG R， VIJAY KUMAR B G， CARNEIRO G， et al. Unsupervised CNN for single view depth estimation： geometry to the rescue［M］∥Computer Vision-ECCV 2016. Cham： Springer， 2016： 740-756. |
| [17] | REN Z， YAN J C， NI B B， et al. Unsupervised deep learning for optical flow estimation［C］∥Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2017， 1495-1501. |
| [18] | GODARD C， AODHA O MAC， BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency［C］∥2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 6602-6611. |
| [19] | ZHANG Y M， CHEN Y M， BAI X， et al. Adaptive unimodal cost volume filtering for deep stereo matching［C］∥Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020， 34（7）： 12926-12934. |
| [20] | POGGI M， TONIONI A， TOSI F， et al. Continual adaptation for deep stereo［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2022， 44（9）： 4713-4729. |
| [21] | TONIONI A， POGGI M， MATTOCCIA S， et al. Unsupervised adaptation for deep stereo［C］∥2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 1614-1622. |
| [22] | TONIONI A， POGGI M， MATTOCCIA S， et al. Unsupervised domain adaptation for depth prediction from images［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（10）： 2396-2409. |
| [23] | WATSON J， AODHA O MAC， TURMUKHAMBETOV D， et al. Learning stereo from single images［C］∥Computer Vision-ECCV 2020. Cham： Springer， 2020： 722-740. |
| [24] | ALEOTTI F， TOSI F， ZHANG L， et al. Reversing the cycle： self-supervised deep stereo through enhanced monocular distillation［C］∥Computer Vision-ECCV 2020. Cham： Springer， 2020： 614-632. |
| [25] | TOSI F， TONIONI A， DE GREGORIO D， et al. NeRF-supervised deep stereo［C］∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 855-866. |
| [26] | MARTIN-BRUALLA R， RADWAN N， SAJJADI M S M， et al. NeRF in the wild： Neural radiance fields for unconstrained photo collections［C］∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 7206-7215. |
| [27] | HUANG L T， BAI J Y， GUO J， et al. On the error analysis of 3D Gaussian splatting and an optimal projection strategy［C］∥Computer Vision-ECCV 2024. Cham： Springer， 2024： 247-263. |
| [28] | MILDENHALL B， SRINIVASAN P P， TANCIK M， et al. NeRF： Representing scenes as neural radiance fields for view synthesis［J］. 2021， 65（1）： 99-106. |
| [29] | HUANG B B， YU Z H， CHEN A P， et al. 2D Gaussian splatting for geometrically accurate radiance fields［C］∥ACM SIGGRAPH 2024 Conference Papers. New York： ACM， 2024： 1-11. |
| [30] | KENDALL A， MARTIROSYAN H， DASGUPTA S， et al. End-to-end learning of geometry and context for deep stereo regression［C］∥2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 66-75. |
| [31] | CHANG J R， CHEN Y S. Pyramid stereo matching network［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 5410-5418. |
| [32] | ZHANG F H， PRISACARIU V， YANG R G， et al. GA-net： guided aggregation net for end-to-end stereo matching［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 185-194. |
| [33] | GUO X Y， YANG K， YANG W K， et al. Group-wise correlation stereo network［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 3268-3277. |
| [34] | SHEN Z L， DAI Y C， RAO Z B. CFNet： cascade and fused cost volume for robust stereo matching［C］∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 13901-13910. |
| [35] | GU X D， FAN Z W， DAI Z Z， et al. Cascade cost volume for high-resolution multi-view stereo and Stereo Matching［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 2495-2504. |
| [36] | LI Z， LIU X， DRENKOW N， et al. Revisiting stereo depth estimation from a sequence-to-sequence perspective with transformers［C］∥2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE Press， 2021： 6197-6206. |
| [37] | XU G， CHENG J， GUO P， et al. Attention concatenation volume for accurate and efficient stereo matchin g［C］∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 12981-12990. |
| [38] | 孔令寅， 朱江平， 应三丛. 基于引导图像和自适应支持域的立体匹配［J］. 光学学报， 2020， 40（9）： 0915001. |
| KONG L Y， ZHU J P， YING S C. Stereo matching based on guidance image and adaptive support region［J］. Acta Optica Sinica， 2020， 40（9）： 0915001 （in Chinese）. | |
| [39] | LIPSON L， TEED Z， DENG J. RAFT-stereo： multilevel recurrent field transforms for stereo matching［C］∥2021 International Conference on 3D Vision （3DV）. Piscataway： IEEE Press， 2022： 218-227. |
| [40] | XU G W， WANG X Q， DING X H， et al. Iterative geometry encoding volume for stereo matching［C］∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 21919-21928. |
| [41] | XU G W， WANG X Q， ZHANG Z X， et al. IGEV++： Iterative multi-range geometry encoding volumes for stereo matching［EB/OL］. （2025-05-11） ［2025-10-01］： . |
| [42] | SCH?NBERGER J L， FRAHM J M. Structure-from-motion revisited［C］∥2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 4104-4113. |
| [43] | SHAH S， DEY D， LOVETT C， et al. AirSim： High-fidelity visual and physical simulation for autonomous vehicles［C］∥Field and Service Robotics. Cham： Springer， 2018： 621-635. |
| [44] | ROMáN R， GONZáLEZ R， TOLEDANO C， et al. Correction of a lunar-irradiance model for aerosol optical depth retrieval and comparison with a star photometer［J］. Atmospheric Measurement Techniques， 2020， 13（11）： 6293-6310. |
| [45] | STUBBS T J， VONDRAK R R， FARRELL W M. A dynamic fountain model for lunar dust［J］. Advances in Space Research， 2006， 37（1）： 59-66. |
| [46] | HOLLINGSWORTH D K， WITTE L C， HINKE J， et al. Reduction in emittance of thermal radiator coatings caused by the accumulation of a Martian dust simulant［J］. Applied Thermal Engineering， 2006， 26（17-18）： 2383-2392. |
| [47] | LIU J Y， ZHANG Q Y， WAN X， et al. LuSNAR： A lunar segmentation， navigation and reconstruction dataset based on Muti-sensor for autonomous exploration［EB/OL］. （2024-09-26）［2025-10-01］： . |
| [48] | HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2008， 30（2）： 328-341. |
| [49] | MARTULL S， PERIS M， FUKUI K. Realistic CG stereo image dataset with ground truth disparity maps： IEICE PRMU-430［R］. Tokyo： IEICE， 2012. |
| [50] | MEI X， SUN X， ZHOU M C， et al. On building an accurate stereo matching system on graphics hardware［C］∥2011 IEEE International Conference on Computer Vision Workshops （ICCV Workshops）. Piscataway： IEEE Press， 2012： 467-474. |
| [51] | SCH?PS T， SCH?NBERGER J L， GALLIANI S， et al. A multi-view stereo benchmark with high-resolution images and multi-camera videos［C］∥2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 2538-2547. |
/
| 〈 |
|
〉 |
CJA