面向投影增强现实跟踪定位器的稳定位姿估计与标定方法

doi:10.7527/S1000-6893.2024.31072

Abstract

Abstract:

To overcome the projection limitations of traditional in-situ Projection Augmented Reality （PAR） systems in occlusion and restricted spaces， a tracking-based PAR assembly assistance system is designed and implemented. Firstly， in view of the transfer target positioning problem existing in system construction， a transfer target positioning algorithm based on voting is proposed by borrowing the idea of “model globally， match locally” from Point Pair Features （PPF）. In addition， in view of the pose calibration problem between the transfer target and the projector， a transfer pose calibration algorithm based on multi-station coupling is proposed. The pose model is simplified by using Kronecker product and solved by constructing Lagrangian equations. Finally， the effectiveness of the proposed algorithm is verified by conducting transfer target positioning stability tests and transfer pose calibration tests. The flexibility of the system is verified by conducting projection positioning tests in the aircraft nose cabin scene. It is shown that the system framework can effectively expand the application scope of the PAR system.

Key words: projection augmented reality, assembly assistance, voting mechanism, pose estimation, iterative method

CLC Number:

Xifu ZHAO, Haihua CUI, Yihua ZHANG, Renchuan YANG, Jiarui ZHOU, Kai PENG, Minjie LIAO. Stable pose estimation and calibration method for projection augmented reality tracking locator[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331072.

Figures/Tables 21

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Table 1

Results of common template method and multi-station coupling method

方法	$R T P$ /mm	$t T P$ /mm
公共模板法 $(M T P) 1$	$- 1.000 1 0.001 1 - 0.005 1 - 0.000 0 - 0.999 5 - 0.013 9 - 0.005 3 - 0.014 0 0.999 7$	$35.211 - 281.034 - 22.938$
多站位耦合法 $(M T P) 2$	$- 0.999 9 0.012 4 - 0.000 1 - 0.012 4 - 0.999 9 - 0.012 0 0.000 0 - 0.012 0 0.999 9$	$35.137 - 281.312 - 22.825$

Table 1

Fig.11

Table 2

Three-dimensional coordinates of three great circles of calibration board in projector coordinate system

点坐标	$p 1 P$ /mm	$p 2 P$ /mm	$p 3 P$ /mm
数值	$- 14.225 - 3.227 456.453$	$- 14.381 - 32.091 464.643$	$30.770 - 3.164 457.588$

Table 2

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

References 22

1	BOTTANI E， VIGNALI G. Augmented reality technology in the manufacturing industry： A review of the last decade［J］. IISE Transactions， 2019， 51（3）： 284-310.
2	LEE J， BAGHERI B， KAO H G. A cyber-physical systems architecture for industry 4.0-based manufacturing systems［J］. Manufacturing Letters， 2015， 3： 18-23.
3	潘志庚，高嘉利，王若楠，等. 面向实物交互的空间增强现实数字孪生法配准技术［J］. 计算机辅助设计与图形学学报， 2021， 33（5）： 655-661.
	PAN Z G， GAO J L， WANG R N， et al. Digital twin registration technique of spatial augmented reality for tangible interaction‍［J］. Journal of Computer-Aided Design & Computer Graphics， 2021， 33（5）： 655-661 （in Chinese）.
4	秦玉波，穆欣伟，邹方. 基于空间增强现实的辅助装配系统研究［J］. 航空制造技术， 2022， 65（12）： 56-62.
	QIN Y B， MU X W， ZOU F. Research on auxiliary assembly system based on spatial augmented reality‍［J］. Aeronautical Manufacturing Technology， 2022， 65（12）： 56-62 （in Chinese）.
5	叶凯玉，周玲，张辉，等. 基于结构光场辅助的面阵定位投影标示技术研发［J］. 航空制造技术， 2024， 67（）： 131-138.
	YE K Y， ZHOU L， ZHANG H， et al. Research on area array projection positioned indication based on structured light field aid‍［J］. Aeronautical Manufacturing Technology， 2024， 67（Sup 1）： 131-138 （in Chinese）.
6	TREINEN T， KOLLA S S V K. Augmented reality for quality inspection， assembly and remote assistance in manufacturing‍［J］. Procedia Computer Science， 2024， 232： 533-543.
7	SERENO M， WANG X Y， BESANCON L， et al. Collaborative work in augmented reality： a survey［J］. IEEE Transactions on Visualization and Computer Graphics， 2022， 28（6）： 2530-2549.
8	BEARDSLEY P， VAN BAAR J， RASKAR R， et al. Interaction using a handheld projector‍［J］. IEEE Computer Graphics and Applications， 2005， 25（1）： 39-43.
9	DEGANI A， LI W B， SACKS R， et al. An automated system for projection of interior construction layouts‍［J］. IEEE Transactions on Automation Science and Engineering， 2019， 16（4）： 1825-1835.
10	CORTES G， MARCHAND E， BRINCIN G， et al. MoSART： Mobile spatial augmented reality for 3D interaction with tangible objects‍［J］. Frontiers in Robotics and AI， 2018， 5： 93.
11	翁金平，程筱胜，崔海华，等. 面向三维测量的空间标志点高稳定实时匹配方法［J］. 激光与光电子学进展， 2016， 53（2）： 109-116.
	WENG J P， CHENG X S， CUI H H， et al. Robust matching algorithm for real-time 3D measurement based on spatial marked points［J］. Laser & Optoelectronics Progress， 2016， 53（2）： 109-116 （in Chinese）.
12	SHI J L， SUN Z X， BAI S Q. 3D reconstruction framework via combining one 3D scanner and multiple stereo trackers‍［J］. The Visual Computer， 2018， 34（3）： 377-389.
13	WEN Z M， WANG Y J， LUO J， et al. Robust， fast and accurate vision-based localization of a cooperative target used for space robotic arm［J］. Acta Astronautica， 2017， 136： 101-114.
14	温卓漫，王延杰，邸男，等. 空间站机械臂位姿测量中合作靶标的快速识别［J］. 航空学报， 2015， 36（4）： 1330-1338.
	WEN Z M， WANG Y J， DI N， et al. Fast recognition of cooperative target used for position and orientation measurement of space station’s robot arm［J］. Acta Aeronautica et Astronautica Sinica， 2015， 36（4）： 1330-1338 （in Chinese）.
15	JIANG T， CHENG X， CUI H， et al. Combined shape measurement based on locating and tracking of an optical scanner［J］. Journal of Instrumentation， 2019， 14（1）： P01006.
16	刘传凯，雷俊雄，刘茜，等. 基于逆投影的多椭圆联合快速高精度定位算法‍［J］. 航空学报， 2023， 44（12）： 327731.
	LIU C K， LEI J X， LIU Q， et al. Fast and precise location measurement of joint multiple ellipses based on inverse transformation of perspective projection‍［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（12）： 327731 （in Chinese）.
17	BARONE S， PAOLI A， RAZIONALE A. 3D reconstruction and restoration monitoring of sculptural artworks by a multi-sensor framework［J］. Sensors， 2012， 12（12）： 16785-16801.
18	JIANG T， CUI H H， CHENG X S. Accurate calibration for large-scale tracking-based visual measurement system［J］. IEEE Transactions on Instrumentation and Measurement， 2020， 70： 5003011.
19	陈媛，罗哉，杨力，等. 一种基于立体靶标定位的三维重建方法［J］. 计量学报， 2024， 45（4）： 471-479.
	CHEN Y， LUO Z， YANG L， et al. A 3D reconstruction method based on stereo target location‍［J］. Acta Metrologica Sinica， 2024， 45（4）： 471-479 （in Chinese）.
20	DROST B， ULRICH M， NAVAB N， et al. Model globally， match locally： Efficient and robust 3D object recognition［C］∥2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2010.
21	BESL P J， MCKAY N D. A method for registration of 3-D shapes［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 1992， 14（2）： 239-256.
22	LEPETIT V， MORENO-NOGUER F， FUA P. EPnP： An accurate O（n） solution to the PnP problem［J］. International Journal of Computer Vision， 2009， 81（2）： 155-166.

[1]	Ping WANG, Hui FU, Guili XU. Camera pose estimation and corresponding points matching based on rotation search [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(2): 326695-326695.
[2]	WANG Zi, SUN Xiaoliang, LI Zhang, CHENG Zilong, YU Qifeng. Transformer based monocular satellite pose estimation [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(5): 325298-325298.
[3]	LIU Fucheng, MU Jinzhen, LIU Zongming, HAN Fei, LI Shuang. Relative pose estimation and optimization of a failure satellite with low-light few-shot images [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(4): 524984-524984.
[4]	MU Jinzhen, LIU Zongming, HAN Fei, ZHOU Yan, LI Shuang. Long-range relative pose estimation and optimization of a failure satellite [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(11): 524959-524959.
[5]	LIU Zongming, MU Jinzhen, ZHANG Shuo, DU Xuan, CAO Shuqing, ZHANG Yu. Visual feature tracking and pose measurement for slow rotating failure satellites [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(1): 524163-524163.
[6]	CHEN Shaochang, HE Huiying, YU Huagang. Constrained Total Least-squares for Source Location Using TDOA Measurements in the Presence of Sensor Position Errors [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2013, 34(5): 1165-1173.
[7]	ZHANG Min, GUO Fucheng, ZHOU Yiyu. A Single Moving Observer Direct Position Determination Method Using a Long Baseline Interferometer [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2013, 34(2): 378-386.
[8]	ZHOU Lai, ZHENG Danli, GU Hongbin, WANG Debao. Research of Visual Interaction for Virtual Reality Flight Training [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2013, 34(10): 2391-2401.
[9]	ZHOU Lai, GU Hongbin, SUN Jin, TANG Yong. Hand Pose Estimation Based on Maximum a Posteriori Framework in Semi-virtual Reality Cockpit [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2011, 32(7): 1252-1259.
[10]	Wang Xiaohong;An Fang;Cao Lijuan;Chen Huaihai. Direct Transformation Method for Mode Synthesis [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2009, 30(1): 92-98.
[11]	Zhang Wu;Luo Shijun. SYNCHRONOUS ITERATIVE METHOD FOR COMPUTATION OF VORTEX FLOWS AT HIGH ANGLES-OF-ATTACK [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 1991, 12(1): 1-5.
[12]	Zhang Baoqin;Lu Zhiliang. PREDICTION OF THE STALL CHARACTERISTICS OF MULTIELEMENT AIRFOILS WITH CONFLUENT BOUNDARY LAYER [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 1988, 9(4): 123-131.
[13]	Zhang Huiliu;Cao Qipeng. CALCULATION OF STEADY TRANSONIC VISCOUS FLOWS WITH SMALL SEPARATION BUBBLE AROUND AN ARBITRARY AIRFOIL [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 1988, 9(3): 103-107.
[14]	Gao Zhenghong. THE TIME-DEPENDENT BODY FITTED COORDINATE GENERATION USING MULTIGRID METHOD [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 1988, 9(3): 184-187.
[15]	Huang Mingke. APPLICATION OF EFFICIENT ITERATION SCHEME AF2 TO COMPUTATIONS OF TRANSONIC FULL-POTENTIAL FLOWS OVER WING-BODY CONBINATIONS [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 1988, 9(1): 11-18.

Stable pose estimation and calibration method for projection augmented reality tracking locator

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 21

References 22

Related Articles 15

Recommended Articles

Metrics

Comments