基于球形变换的无人机视频图像实时拼接方法

doi:10.7527/S1000-6893.2023.28364

Abstract

Abstract:

In the emergency situations such as fire situation monitoring， the image of the task area is obtained through UAV aerial photography. The traditional UAV image mosaic method has the problem of low timeliness. However， the real-time splicing based on video suffers from problem of homography error accumulation and splicing interruption. This paper designs and implements a real-time video image stitching method for UAV based on spherical transformation. After spherical transformation is added， global consistency can be increased for large angle stitching. In addition， only a single homography matrix is calculated， which ensures the splicing efficiency of the algorithm on the premise of angle robustness. This paper first calculates the SURF feature description of the generated image， uses the matching algorithm to match the Speeded Up Robust Features（SURF）features between consecutive images， and then uses the random sampling consistency algorithm to screen the matching. According to the matching relationship between images， the homography transformation matrix is calculated， and the error correction sphere parameters are calculated after the homography matrix is split. After the spherical transformation correction， homography is used to complete image mosaic， and finally image fusion is carried out. The experimental results show that the method proposed in this paper can improve the timeliness of traditional image mosaic（Compared with the Shape-Preserving Half-Projective （SPHP） algorithm， the tie splicing time has been reduced from 2 345.25 ms to 1 528.6 ms， resulting in a 34.8% improvement in algorithm efficiency） and the robustness of image splicing with the change of camera angle and it can still perform well in situations with large angles and poor visual perception.

Key words: UAV, image stitching, SURF, spherical transformation, homography transformation

CLC Number:

Guoqi ZENG, Zifan NIU, Lili ZHENG, Jie LI, Delin HAO. A real time video image stitching method for UAV based on spherical transformation[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(24): 328364.

Figures/Tables 23

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Table 1

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig. 14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Fig.20

Table 2

Table 3

References 20

1	MORANDO L， RECCHIUTO C T， CALLA J， et al. Thermal and visual tracking of photovoltaic plants for autonomous UAV inspection［J］. Drones， 2022， 6（11）： 347.
2	ZHANG Y T， CHEN S， WANG W Y， et al. Pyramid attention based early forest fire detection using UAV imagery［J］. Journal of Physics： Conference Series， 2022， 2363（1）： 012021.
3	CARVAJAL-RAMÍREZ F， SERRANO J M P R， AGÜERA-VEGA F， et al. A comparative analysis of phytovolume estimation methods based on UAV-photogrammetry and multispectral imagery in a Mediterranean forest［J］. Remote Sensing， 2019， 11（21）： 2579.
4	SKONDRAS A， KARACHALIOU E， TAVANTZIS I， et al. UAV mapping and 3D modeling as a tool for promotion and management of the urban space［J］. Drones， 2022， 6（5）： 115.
5	SHAO R Z， DU C， CHEN H， et al. Fast anchor point matching for emergency UAV image stitching using position and pose information［J］. Sensors， 2020， 20（7）： 2007.
6	ABBADI N K E， HASSANI S A AL， ABDULKHALEQ A H. A review over panoramic image stitching techniques［J］. Journal of Physics： Conference Series， 2021， 1999（1）： 012115.
7	LOWE D G. Object recognition from local scale-invariant features［C］∥ Proceedings of the Seventh IEEE International Conference on Computer Vision. Piscataway： IEEE Press， 2002： 1150-1157.
8	BAY H， ESS A， TUYTELAARS T， et al. Speeded-up robust features （SURF）［J］. Computer Vision and Image Understanding， 2008， 110（3）： 346-359.
9	FISCHER P， DOSOVITSKIY A， BROX T. Descriptor matching with convolutional neural networks： A comparison to SIFT［J］. Computer Science， 2014， 4（7）： 678-694.
10	NGUYEN T， CHEN S W， SHIVAKUMAR S S， et al. Unsupervised deep homography： A fast and robust homography estimation model［J］. IEEE Robotics and Automation Letters， 2018， 3（3）： 2346-2353.
11	ZHANG Y J， MEI X G， MA Y， et al. Hyperspectral panoramic image stitching using robust matching and adaptive bundle adjustment［J］. Remote Sensing， 2022， 14（16）： 4038.
12	ZARAGOZA J， CHIN T J， BROWN M S， et al. As-projective-as-possible image stitching with moving DLT［C］∥ 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2013： 2339-2346.
13	CHANG C H， SATO Y， CHUANG Y Y. Shape-preserving half-projective warps for image stitching［C］∥ 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2014： 3254-3261.
14	CHEN Y S， CHUANG Y Y. Natural image stitching with the global similarity prior［M］∥ Computer Vision⁃ECCV 2016. Cham： Springer International Publishing， 2016： 186-201.
15	ZHANG G F， HE Y， CHEN W F， et al. Multi-viewpoint panorama construction with wide-baseline images［C］∥ IEEE Transactions on Image Processing. Piscataway： IEEE Press， 2016： 3099-3111.
16	LIN K M， JIANG N J， LIU S C， et al. Direct photometric alignment by mesh deformation［C］∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 2701-2709.
17	LEE K Y， SIM J Y. Warping residual based image stitching for large parallax［C］∥ 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 8195-8203.
18	DE A LIMA NETO E， RODRIGUES P C. Kernel robust singular value decomposition［J］. Expert Systems With Applications， 2023， 211： 118555.
19	LI X， LIU Y R， LI D H， et al. Spherical image stitching based on ORB and PROSAC algorithm［C］∥ Proceedings of the 3rd International Conference on Intelligent Information Processing. New York： ACM， 2018： 160-165.
20	KIM T， IM Y J. Automatic satellite image registration by combination of matching and random sample consensus［J］. IEEE Transactions on Geoscience and Remote Sensing， 2003， 41（5）： 1111-1117.

算法	特征/个	匹配点/个	运行时间/ms	千点运行时间/ms
ORB	500	161	345.2	690.6
SURF	796	398	1 726.48	2 168

算法	耗时/ms	平均耗时/ms	输出大小/KB
SPHP	351 788	2 345.25	3 915
ODM	858 904	5 726.48	4 330
Ball	285 900	1 906.11	3 860

算法	耗时/ms	平均耗时/ms	输出大小/KB
SPHP	351 788	2 345.25	3 915
ODM	858 904	5 726.48	4 330
Ball	229 290	1 528.60	3 817

[1]	Xiangyu YU, Wen LI, Jie YAN, Shizhe LIANG. Simulation research on thermal management system of fuel cell for liquid hydrogen powered UAV [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 630964-630964.
[2]	Pengqian YANG, Yutong CHEN, Junhui LIU, Jiehao YANG, Jiayuan SHAN, Shijun SUN. Aerodynamic and operational characteristics analysis for tandem wing cargo UAV at high angle of attack [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 131056-131056.
[3]	Rongzu LI, Li LIU, Dun YANG. Optimal design of hydrogen-powered UAV based on multi-source domain fusion surrogate model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 630979-630979.
[4]	Kaifang WAN, Zhilin WU, Yunhui WU, Haozhi QIANG, Yibo WU, Bo LI. Cooperative location of multiple UAVs with deep reinforcement learning in GPS-denied environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331024-331024.
[5]	Lingfeng JIANG, Xinkai LI, Hai ZHANG, Hanwei LI, Hongli ZHANG. Mapless navigation of UAVs in dynamic environments based on an improved TD3 algorithm [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331035-331035.
[6]	Xilun DING, Yitong CHEN, Chengcai WANG, Kun XU. Research status and prospect of space robot operation technology [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(6): 531556-531556.
[7]	Ruicong ZHANG, Tianyu WANG, Yurong HE, Jiaqi ZHU, Jiecai HAN. Research progress and application perspectives on smart window scattering control technology [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(6): 531844-531844.
[8]	Jinwu XIANG, Kai MA, Zi KAN, Daochun LI, Kexin ZHENG, Hanxuan CHEN. Review of key technologies for hydrogen powered unmanned aerial vehicles [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(5): 531603-531603.
[9]	Kaishang LI, Haoqi FAN, Runzi WANG, Xiancheng ZHANG, Shantung TU. Life design method of high-temperature equipment based on dual-scale damage theory [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(5): 531706-531706.
[10]	Jinxiao CHEN, Peitang WEI, Yanjun LI, Huaiju LIU, Caichao ZHU. Experiment and anti-scuffing design for aviation gear scuffing based on material, surface treatment and lubricant combinations [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(4): 430777-430777.
[11]	Qishuai DING, Bangjun LEI, Zhengping WU. A lightweight single object tracking algorithm for UAVs based on Siamese network [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(4): 330925-330925.
[12]	Qulong WEI, Lihong JIANG, Zheng LIU, Lin ZHU, Lianhai FAN, Guangang WANG, Mingjie ZHAO, Zhenghua GUO. Lattice structure and mechanical properties of TPMS prepared by selective laser melting [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 430593-430593.
[13]	Nuo MA, Shechun WEI, Junhui MENG, Qingyang LIU, Yusheng LEI. Flow field characteristics and dynamics of internal supply chamber separating from UAV considering effect of deceleration parachutes [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 130755-130755.
[14]	Yiquan WU, Kang TONG. Research advances on deep learning-based small object detection in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 30848-030848.
[15]	Kai YANG, Mengfei ZHANG, Chongguang SHI, Yaokun YU, Xiaogang ZHENG, Chengxiang ZHU, Yancheng YOU. Method of three-dimensional curved stream-surface and its application in external waverider [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(2): 130492-130492.

A real time video image stitching method for UAV based on spherical transformation

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 23

References 20

Related Articles 15

Recommended Articles

Metrics

Comments