手姿态估计是半虚拟现实(SVR)座舱环境中进行人机交互(HCI)的基础工作,为此提出一种基于最大后验概率(MAP)框架的手姿态估计方法。针对手部自遮挡问题,建立多视点手势特征树,通过树节点搜索实现手姿态参数的快速估计,并利用时序信息提高估计精度。在树节点搜索过程中,采用改进的局部敏感哈希(LSH)索引算法提高搜索效率。实验结果表明,该方法得到的姿态参数能实时、准确地驱动虚拟手,再现用户真实手的各种动作和状态。
To realize natural interaction in a semi-virtual reality (SVR) system, hand pose estimation is a crucial step of human-computer interaction (HCI) in a semi-virtual reality cockpit. In order to enhance the sense of immersion for the pilot in the semi-virtual reality cockpit, a hand pose estimation method based on maximum a posteriori (MAP) framework is proposed in this paper. Under the MAP framework, a multi-view tree that contains information of multiple cameras is put forward to deal with the self-occlusion in hand motion. Hand gesture parameters are estimated by searching among the tree nodes. In this method, temporal consistency is used to advance the estimated accuracy, and locality sensitive hashing (LSH) is improved to raise the efficiency in the tree searching algorithm. Experimental results show that the proposed algorithm possesses the characteristic of real-time and accurate rendering of the virtual hand and the ability to reconstruct hand postures in a semi-virtual reality cockpit.
[1] Gu H B, Wu D S, Liu H. Development of a novel low-cost flight simulator for pilot training[J]. World Academy of Science, Engineering and Technology, 2009(60): 685-689.
[2] Stenger B, Mendonca P R S, Cipolla R. Model-based 3D tracking of an articulated hand//Proceedings of 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2001: 310-315.
[3] Lu S, Metaxas D, Samaras D, et al. Using multiple cues for hand tracking and model refinement//Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2003: 443-450.
[4] Lin J, Wu Y, Huang T S. 3D model-based hand tracking using stochastic direct search method//Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition. 2004: 693-698.
[5] Rosales R, Athitsos V, Sclaroff S. 3D hand pose reconstruction using specialized mappings//Proceedings of 8th International Conference on Computer Vision. 2001: 378-385.
[6] Wu Y, Lin J, Huang T S. Analyzing and capturing articulated hand motion in image sequences[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(12): 1910-1922.
[7] 刘棠丽, 吴心筱, 梁玮, 等. 基于非参数信念传播的可行C-空间关节人手跟踪方法[J]. 计算机辅助设计与图形学学报, 2008, 20(4): 476-481. Liu Tangli, Wu Xinxiao, Liang Wei, et al. 3D articulated hand tracking by nonparametric belief propagation on feasible configuration space[J]. Journal of Computer-Aided Design & Computer Graphics, 2008, 20(4): 476-481. (in Chinese)
[8] 冯志全, 杨波, 李毅, 等. 以时间优化为目标的粒子滤波手势跟踪方法研究[J]. 电子学报, 2009, 37(9): 1989-1995. Feng Zhiquan, Yang Bo, Li Yi, et al. Research on hand gestures tracking based on particle filtering aiming at optimizing time cost[J]. Acta Electronica Sinica, 2009, 37(9): 1989-1995. (in Chinese)
[9] Delamarre Q, Faugeras O. 3D articulated models and multi-view tracking with physical forces[J]. Computer Vision and Image Understanding, 2001, 81(3): 328-357.
[10] Ueda E, Matsumoto Y, Imai M, et al. A hand-pose estimation for vision-based human interfaces[J]. IEEE Transactions on Industrial Electronics, 2003, 50(4): 676-684.
[11] Hamada Y, Shimada N, Shirai Y. Hand shape estimation using sequence of multiple viewpoint images based on transition network[J]. Electronics and Communications in Japan, 2004, 87(8): 33-42.
[12] Guan H, Chang J S, Chen L B, et al. Multi-view appearance-based 3D hand pose estimation//Proceedings of Computer Vision and Pattern Recognition Workshop. 2006: 154-159.
[13] Romero J, Kjellstrom H, Kragic D. Hands in action: real-time 3D reconstruction of hands in interacton with objects//Proceedings of the 2010 IEEE International Conference on Robotics and Automation. 2010: 458-463.
[14] Romero J, Kjellstrom H, Kragic D. Monocular real-time 3D articulated hand pose estimation// Proceedings of the 9th IEEE-RAS International Conference on Humanoid Robots. 2009: 87-92.
[15] 汤勇, 顾宏斌, 周来. 交互系统中实时手势分割及指尖检测方法[J]. 光电工程, 2010, 37(7): 145-150. Tang Yong, Gu Hongbin, Zhou Lai. Real-time hand segmentation and fingertip detection for interaction[J]. Opto-Electronic Engineering, 2010, 37(7): 145-150. (in Chinese)
[16] Dalal N, Triggs B. Histograms of oriented gradients for human detection//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005: 886-893.
[17] Duda R O, Hart P E, Stork D G. 模式分类[M]. 李宏东, 姚天翔, 译. 2版. 北京: 机械工业出版社, 2003: 435-437. Duda R O, Hart P E, Stork D G. Pattern classification[M]. Li Hongdong, Yao Tianxiang, translated. 2nd ed. Beijing: China Machine Press, 2003: 435-437. (in Chinese)
[18] Datar M, Immorlica N, Indyk P. Locality-sensitive hashing scheme based on p-stable distributions//Proceedings of the 20th Annual Symposium on Computational Geometry. 2004: 253-262.
[19] Lv Q, Josephson W, Wang Z, et al. Multi-probe LSH: efficient indexing for high-dimensional similarity search// Proceedings of the 33rd International Conference on Very Large Data Bases. 2007: 950-961.
CJA