基于轻量级神经网络的非合作目标位姿单目测量

doi:10.7527/S1000-6893.2024.30248

Abstract

Abstract:

Estimating the pose of non-cooperative targets from monocular images stands as a pivotal technology for future space missions. Recent advancements in deep neural networks have surpassed traditional methods in pose measurement accuracy. However， these networks often entail a high number of parameters and significant computational complexity. This poses a challenge for deployment in on-orbit applications where real-time measurement is crucial， as computational resources are limited. Reducing the number of network parameters compromises the ability to extract representative features， leading to degraded pose estimation performance. To tackle this problem， we present an approach using lightweight neural networks that maintains high accuracy in pose estimation， which is a task far from trivial. Our solution involves a novel semantic keypoint localization method. We develop a lightweighted neural network model with a mere 1.1 ×10⁶ parameters. To enhance the precision of semantic keypoint localization and subsequent pose estimation， we introduce a heatmap decoding technique that allows for sub-pixel level accuracy， while enabling end-to-end supervision of semantic keypoint localization. Moreover， we develop an auxiliary layer supervised training method to further refine the accuracy of semantic keypoint localization. Experiments on public datasets demonstrate that our method not only achieves the highest pose measurement accuracy among all lightweight models with fewer than 10⁷ parameters， but also sets a new benchmark. Additionally， tests on embedded development boards reveal that our method attains measurement frequencies of 5 Hz and 11 Hz in 10 W and 30 W power modes， respectively.

Key words: pose measurement, monocular image, lightweight, deep learning, semantic keypoint

CLC Number:

V248.1

Zi WANG, Jinghao WANG, Yang LI, Zhang LI, Qifeng YU. Non-cooperative target pose estimation from monocular images based on lightweight neural network[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(22): 330248.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Table 1

Parameters setting of heatmap head

操作	输入特征尺寸	输出特征尺寸	卷积核尺寸
卷积层1	$c × h × w$	$4 N × h × w$	$3 × 3$
BN层	$4 N × h × w$	$4 N × h × w$
ReLU激活层	$4 N × h × w$	$4 N × h × w$
卷积层2	$4 N × h × w$	$N × h × w$	$1 × 1$

Table 1

Fig.4

Fig.5

Table 2

Sensitivity analysis of the parameter K

K	3	5	8	12	16	21
得分/ $10 - 3$	15.9	16.5	15.5	15.2	13.7	17.1
成功率/‰	65.5	73.0	76	95.5	103.0	85.5

Table 2

Table 3

Analysis of ablation study

组件	模型1	模型2	模型3	模型4	模型5
辅助层		√		√	√
回归损失			√	√	√
非线性优化					√
得分/ $10 - 3$	25.9	26.0	22.5	19.4	13.7
成功率/‰	8.0	13.5	29.5	35.5	103.0

Table 3

Fig.6

Table 4

Comparison with SOTA methods

指标	实验1	实验2	实验3	实验4	实验5	实验6	均值	SPN^［12］	文献［18］	文献［19］
$E R$ 均值/（°）	0.62	0.64	0.58	0.67	0.59	0.61	0.62	8.43	0.99	0.66
$E R$ 中值/（°）	0.40	0.39	0.40	0.40	0.38	0.40	0.40	7.07	0.80	0.49
$E$ / $10 - 3$	13.7	14.3	15.1	17.9	13.1	16.3	15.1		22.9	15.3
$t - t ̂$ 均值/mm	2.5	2.4	2.3	2.4	2.5	2.5	2.4	55.0	3.8	2.6
	2.7	2.4	2.4	2.4	2.6	2.5	2.5	46.0	3.7	2.5
	31.6	31.5	28.9	30.0	32.3	31.3	30.9	78.0	64.4	37.4
$t - t ̂$ 中值/mm	1.7	1.7	1.6	1.5	1.6	1.6	1.6	24.0	2.9	1.8
	1.7	1.7	1.6	1.5	1.7	1.6	1.6	21.0	2.7	1.6
	16.0	15.8	16.6	15.1	16.5	15.9	16.0	496.0	43.7	21.4

Table 4

Fig.7

Fig.8

Table 5

Table 6

References 26

1	王楷，徐世杰，黎康，等. 双视线测量相对导航方法误差分析与编队设计［J］. 航空学报， 2018， 39（9）： 322014.
	WANG K， XU S J， LI K， et al. Error analysis and formation design for double line-of-sight measuring relative navigation method［J］. Acta Aeronautica et Astronautica Sinica， 2018， 39（9）： 322014 （in Chinese）.
2	肖余之，靳永强，陈欢龙，等. 在轨服务若干关键技术研究进展［J］.上海航天（中英文）， 2021， 38（3）： 85-95.
	XIAO Y Z， JIN Y Q， CHEN H L， et al. Research progress on several key technologies of on-orbit service ［J］. Aerospace Shanghai （Chinese & English）， 2021， 38（3）： 85-95 （in Chinese）.
3	于浛，魏喜庆，宋申民，等. 基于自适应容积卡尔曼滤波的非合作航天器相对运动估计［J］. 航空学报， 2014， 35（8）： 2251-2260.
	YU H， WEI X Q， SONG S M， et al. Relative motion estimation of non-cooperative spacecraft based on adaptive CKF［J］. Acta Aeronautica et Astronautica Sinica， 2014， 35（8）： 2251-2260 （in Chinese）.
4	OPROMOLLA R， FASANO G， RUFINO G， et al. A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations［J］. Progress in Aerospace Sciences， 2017， 93： 53-72.
5	KISANTAL M， SHARMA S， PARK T H， et al. Satellite pose estimation challenge： dataset， competition design， and results［J］. IEEE Transactions on Aerospace and Electronic Systems， 2020， 56（5）： 4083-4098.
6	PARK T H， MÄRTENS M， JAWAID M， et al. Satellite pose estimation competition 2021： results and analyses［J］. Acta Astronautica， 2023， 204： 640-665.
7	WANG Z， CHEN M L， GUO Y L， et al. Bridging the domain gap in satellite pose estimation： a self-training approach based on geometrical constraints［J］. IEEE Transactions on Aerospace and Electronic Systems， 2024， 60（3）： 2500-2514.
8	CHEN B， CAO J W， PARRA A， et al. Satellite pose estimation with deep landmark regression and nonlinear pose refinement［C］∥ 2019 IEEE/CVF International Conference on Computer Vision Workshop （ICCVW）. Piscataway： IEEE Press， 2019： 2816-2824.
9	LEPETIT V， MORENO-NOGUER F， FUA P. EPnP： an accurate O（n） solution to the PnP problem［J］. International Journal of Computer Vision， 2009， 81（2）： 155-166.
10	BECHINI M， GU G， LUNGHI P， et al. Robust spacecraft relative pose estimation via CNN-aided line segments detection in monocular images［J］. Acta Astronautica， 2024， 215： 20-43.
11	PROENÇA P F， GAO Y. Deep learning for spacecraft pose estimation from photorealistic rendering［C］‍∥ 2020 IEEE International Conference on Robotics and Automation （ICRA）. Piscataway： IEEE Press， 2020： 6007-6013.
12	SHARMA S， D’AMICO S. Neural network-based pose estimation for noncooperative spacecraft rendezvous［J］. IEEE Transactions on Aerospace and Electronic Systems， 2020， 56（6）： 4638-4658.
13	PARK T H， SHARMA S， D’AMICO S. Towards robust learning-based pose estimation of noncooperative spacecraft［C］‍∥ 2019 AAS/AIAA Astrodynamics Specialist Conference. 2019.
14	SANDLER M， HOWARD A， ZHU M L， et al. MobileNetV2： inverted residuals and linear bottlenecks［C］‍∥ 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 4510-4520.
15	ZHANG X Y， ZHOU X Y， LIN M X， et al. ShuffleNet： an extremely efficient convolutional neural network for mobile devices［C］∥ 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 6848-6856.
16	褚晶辉，李梦，吕卫. 基于深度学习的自适应动态滤波器剪枝方法［J］. 激光与光电子学进展， 2022， 59（24）： 2415003.
	CHU J H， LI M， LÜ W. Adaptive dynamic filter pruning approach based on deep learning［J］. Laser & Optoelectronics Progress， 2022， 59（24）： 2415003 （in Chinese）.
17	黄赟，张帆，郭威，等. 一种基于数据标准差的卷积神经网络量化方法［J］. 电子学报， 2023， 51（3）： 639-647.
	HUANG Y， ZHANG F， GUO W， et al. A quantification method of convolutional neural network based on data standard deviation［J］. Acta Electronica Sinica， 2023， 51（3）： 639-647 （in Chinese）.
18	王梓，孙晓亮，李璋，等．基于Transformer模型的卫星单目位姿估计方法［J］．航空学报， 2022， 43（1）： 325298.
	WANG Z， SUN X L， LI Z， et al. Transformer based monocular satellite pose estimation［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（1）： 325298 （in Chinese）.
19	WANG Z， ZHANG Z， SUN X L， et al. Revisiting monocular satellite pose estimation with transformer［J］. IEEE Transactions on Aerospace and Electronic Systems， 2022， 58（5）： 4279-4294.
20	POSSO J， BOIS G， SAVARIA Y. Mobile-URSONet： an embeddable neural network for onboard spacecraft pose estimation［C］∥ 2022 IEEE International Symposium on Circuits and Systems （ISCAS）. Piscataway： IEEE Press， 2022： 794-798.
21	BLACK K， SHANKAR S， FONSEKA D， et al. Real-time， flight-ready， non-cooperative spacecraft pose estimation using monocular imagery［C］‍∥ 31st AAS/AIAA Space Flight Mechanics Meeting， 2021.
22	LOTTI A， MODENINI D， TORTORA P， et al. Deep learning for real-time satellite pose estimation on tensor processing units［J］. Journal of Spacecraft and Rockets， 2023， 60（3）： 1034-1038.
23	于起峰，尚洋. 摄像测量学原理与应用研究［M］. 北京：科学出版社， 2009： 23-32.
	YU Q F， SHANG Y. Videometrics： principles and researches［M］. Beijing： Science Press， 2009： 23-32 （in Chinese）.
24	FISCHLER M. Random sample consensus： a paradigm for model fitting with applications to image analysis and automated cartography［J］. Communications of the ACM， 1981， 24（6）： 381-395.
25	YU C Q， XIAO B， GAO C X， et al. Lite-HRNet： a lightweight high-resolution network［C］‍∥ 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 10435-10445.
26	WANG X Y， BO L F， LI F X. Adaptive wing loss for robust face alignment via heatmap regression［C］‍∥ 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2019： 6970-6980.

方法	得分	参数量/10⁶	测量频率/Hz
UniAdelaide^［8］	9.4	28.5
SLAB_baseline^［13］	62.6	5.6
文献［22］ Lite4	14.3	12.1	1.7
文献［22］ Lite3	15.2	7.2	2.9
文献［22］ Lite2	16.3	5.1	4.1
文献［22］ Lite1	16.6	4.4	5.4
文献［22］ Lite0	18.6	3.6	7.7
文献［21］	40.9	3.6	11.9
本文方法	15.1	1.1	10.9

[1]	Jianyu XU, Li ZHOU, Zhanxue WANG, Jie SHI, Hao SHI. Calculation method for hypersonic plume infrared radiation based on a fast line-by-line calculation model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 630778-630778.
[2]	Lingjie MENG, Hongguang LI, Xinjun LI. SAR image simulation method guided by geomorphic category information [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(7): 331003-331003.
[3]	Qishuai DING, Bangjun LEI, Zhengping WU. A lightweight single object tracking algorithm for UAVs based on Siamese network [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(4): 330925-330925.
[4]	Zhihao ZHAO, Zhaohua YANG, Yun WU, Yuanjin YU. Single-photon counting imaging denoising method based on deep learning in low-light environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630531-630531.
[5]	Yiquan WU, Kang TONG. Research advances on deep learning-based small object detection in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 30848-030848.
[6]	Jihong ZHU, Jiacheng HAN, Xiaojun GU, Yahui ZHANG, Jun WANG, Jie HOU, Weihong ZHANG. Advances and challenges in cross-domain vehicle structures and morphing configuration design technologies [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 431686-431686.
[7]	Xiaowei JIANG, Yiquan WU. Research progress of UAV aerial image mosaic methods [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(17): 331799-331799.
[8]	Lin CHEN, Xiwen GU, Zhiying CHEN, Zhuo ZHANG, Xiaoliang SUN. High-precision monocular vision pose measurement for large distance span in carrier landing guidance [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(15): 331568-331568.
[9]	Bin SUN, Hang YOU, Wenbo LI, Xiangrui LIU, Jiayi MA. Dual-band payload image fusion and its applications in low-altitude remote sensing [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531343-531343.
[10]	Lin CHEN, Qing ZHU, Han HU, Yulin DING, Pengxin GU. FLASH: Flexible and lightweight awareness of slope hazard [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531293-531293.
[11]	Fanteng MENG, Yong QIN, Jing CUI, Yunpeng WU, Zicheng ZHANG, Shaowei WEI. Unknown risk detection in external environment of railroad using UAV images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531262-531262.
[12]	Weishi CHEN, Hongchuang NIU, Xin WANG, Jian WAN, Xianfeng LU, Jie ZHANG, Qingbin WANG. Review on multi-source detection technologies for birds and drones in airport clearance area [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 31251-031251.
[13]	Jie LIN, Zhigong TANG, Weiqi QIAN, Yueqing WANG, Peng ZHANG, Weixia XU, Jie LIU. Research progress and prospects of aircraft aerodynamic design based on generative models [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 631679-631679.
[14]	Yonghai WANG, Haoge LI, Jiaxin LI, Yi DUAN, Chuan TIAN, Lingxi GUO, Xusheng WU. Rapid aircraft shape generation based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 631614-631614.
[15]	Jiaqi LIU, Rongqian CHEN, Jinhua LOU, Xu HAN, Hao WU, Yancheng YOU. Aerodynamic shape optimization of high-speed helicopter rotor airfoil based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(9): 529828-529828.

Non-cooperative target pose estimation from monocular images based on lightweight neural network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 26

Related Articles 15

Recommended Articles

Metrics

Comments

功率/W	线程数	耗时/ms	得分
10	2	197.6	13.7
15	4	184.1	13.7
30	2	111.1	13.7
30 （Max）	8	92.3	13.7