高性能流场并行粒子追踪数据管理系统

杨昌和; 李彦达; 张江; 王昉; 袁晓如

doi:10.7527/S1000-6893.2021.25748

航空学报 >

2021 , Vol. 42 >Issue 9: 625748 - 625748

DOI: https://doi.org/10.7527/S1000-6893.2021.25748

国家数值风洞(NNW)进展及应用专栏

高性能流场并行粒子追踪数据管理系统

杨昌和 ,
李彦达 ,
张江 ,
王昉 ,
袁晓如

展开

1. 北京大学信息科学技术学院机器感知与智能教育部重点实验室, 北京 100871;
2. 北京大学大数据分析与应用技术国家工程实验室, 北京 100871;
3. 北京大学, 北京 100871;
4. 中国空气动力研究与发展中心计算空气动力研究所, 绵阳 621000

收稿日期: 2021-03-30

修回日期: 2021-05-06

网络出版日期: 2021-05-24

基金资助

国家数值风洞工程

收起

High-performance flow parallel particle tracing data management system

YANG Changhe ,
LI Yanda ,
ZHANG Jiang ,
WANG Fang ,
YUAN Xiaoru

Expand

1. Key Laboratory of Machine Perception and Intelligence, Ministry of Education, School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, China;
2. National Engineering Laboratory of Big Data Analysis and Application Technology, Peking University, Beijing 100871, China;
3. Peking University, Beijing 100871, China;
4. Computational Aerodynamics Institute, China Aerodynamics Research and Development Center, Mianyang 621000, China

Received date: 2021-03-30

Revised date: 2021-05-06

Online published: 2021-05-24

Supported by

National Numerical Windtunnel Project

Fold

摘要

随着当下计算能力和存储性能的提升，流场数据产出的规模越来越庞大，针对流场数据的可视化应用对于硬件及软件算法的要求也随之提高。基于国家数值风洞（NNW）工程支持，主导设计并开发了高性能流场并行粒子追踪数据管理系统，帮助用户探索和分析大规模流场数据。该系统针对流场数据提供多种高效的数据管理方法，在超算集群上针对并行粒子追踪过程进行了数据预取优化与负载均衡优化。对于粒子追踪过程中产生的流线（或迹线）及进程工作记录数据，该系统支持用户在本地平台上进行性能诊断和分析。使用不同流场数据集开展的两个应用实例验证了该系统的有效性。

关键词： 可视化; 科学可视化; 流场可视化; 并行粒子追踪; 国家数值风洞(NNW)工程; 数据管理; 可视分析

本文引用格式

杨昌和 , 李彦达 , 张江 , 王昉 , 袁晓如 . 高性能流场并行粒子追踪数据管理系统[J]. 航空学报, 2021 , 42(9) : 625748 -625748 . DOI: 10.7527/S1000-6893.2021.25748

Abstract

With the current improvement in computing power and storage performance, the scale of flow data output is getting larger and larger, and the requirements for hardware and software algorithms for visualization applications of flow data have also increased. Supported by the National Numerical Windtunnel (NNW) Project, a high-performance flow parallel particle tracking data management system is developed to help users explore and analyze large-scale flow field data. The system provides a variety of efficient data management methods for flow data, and optimizes data prefetching and load balancing in the process of parallel particle tracing on supercomputer clusters. For the streamline (or pathline) and process work record data generated in the particle tracking process, the system supports users to conduct performance diagnosis and analysis on the local platform. Two cases using different flow field data sets verify the effectiveness of the system proposed.

Key words： visualization; scientific visualization; flow visualization; parallel particle tracing; National Numerical Windtunnel (NNW) Project; data management; visual analysis

参考文献

[1] 陈坚强. 国家数值风洞工程(NNW)关键技术研究进展[J/OL]. (2021-04-28)[2021-05-05]. 中国科学:技术科学, https://kns.cnki.net/kcms/detail/11.5844.TH.2021-0428.0914.006.html. CHEN J Q. Advances in the key technologies of Chinese National Numerical Wind Tunnel Project[J/OL]. (2021-04-28)[2021-05-05]. Scientia Sinica Technologica, https://kns.cnki.net/kcms/detail/11.5844.TH.2021-0428.0914.006.html (in Chinese).
[2] LARAMEE R S, HAUSER H, DOLEISCH H, et al. The state of the art in flow visualization:dense and texture-based techniques[J]. Computer Graphics Forum, 2004, 23(2):203-221.
[3] MCLOUGHLIN T, LARAMEE R S, PEIKERT R, et al. Over two decades of integration-based, geometric flow visualization[J]. Computer Graphics Forum, 2010, 29(6):1807-1829.
[4] POST F H, VROLIJK B, HAUSER H, et al. The state of the art in flow visualisation:feature extraction and tracking[J]. Computer Graphics Forum, 2003, 22(4):775-792.
[5] BRUCKSCHEN R, KUESTER F, HAMANN B, et al. Real-time out-of-core visualization of particle traces[C]//Proceedings of the IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics. Piscataway:IEEE Press, 2001:45-50.
[6] ELLSWORTH D, GREEN B, MORAN P. Interactive terascale particle visualization[C]//IEEE Visualization 2004. Piscataway:IEEE, 2004:353-360.
[7] CABRAL B, LEEDOM L C. Imaging vector fields using line integral convolution[C]//Proceedings of SIGGRAPH 1993. New York:ACM Press, 1993:263-270.
[8] SHEN H W, KAO D L. UFLIC:A line integral convolution algorithm for visualizing unsteady flows[C]//Proceedings of the 8th Conference on Visualization. Washington, D.C.:IEEE Computer Society Press, 1997:317-322.
[9] HALLER G. Distinguished material surfaces and coherent structures in three-dimensional fluid flows[J]. Physica D:Nonlinear Phenomena, 2001, 149(4):248-277.
[10] GARTH C, GERHARDT F, TRICOCHE X, et al. Efficient computation and visualization of coherent structures in fluid flow applications[J]. IEEE Transactions on Visualization and Computer Graphics, 2007, 13(6):1464-1471.
[11] EDMUNDS M, LARAMEE R S, CHEN G N, et al. Surface-based flow visualization[J]. Computers & Graphics, 2012, 36(8):974-990.
[12] KENDALL W, WANG J Y, ALLEN M, et al. Simplified parallel domain traversal[C]//Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. Piscataway:IEEE, 2011:1-11.
[13] BLUMOFE R D, LEISERSON C E. Scheduling multithreaded computations by work stealing[C]//Proceedings 35th Annual Symposium on Foundations of Computer Science. Piscataway:IEEE, 1994:356-368.
[14] DINAN J, LARKINS D B, SADAYAPPAN P, et al. Scalable work stealing[C]//Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. New York:ACM Press, 2009:11.
[15] PUGMIRE D, CHILDS H, GARTH C, et al. Scalable computation of streamlines on very large datasets[C]//Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. New York:ACM Press, 2009:12.
[16] LU K W, SHEN H W, PETERKA T. Scalable computation of stream surfaces on large scale vector fields[C]//Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Piscataway:IEEE, 2014:1008-1019.
[17] MVLLER C, CAMP D, HENTSCHEL B, et al. Distributed parallel particle advection using work requesting[C]//2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV). Piscataway:IEEE, 2013:1-6.
[18] MOROZOV D, PETERKA T. Efficient delaunay tessellation through k-d tree decomposition[C]//Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Piscataway:IEEE, 2016:728-738.
[19] ZHANG J, GUO H Q, HONG F, et al. Dynamic load balancing based on constrained k-d tree decomposition for parallel particle tracing[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(1):954-963.
[20] RHODES P J, TANG X, BERGERON R D, et al. Iteration aware prefetching for large multidimensional datasets[C]//Proceedings of the 17th International Conference on Scientific and Statistical Database Management. Los Alamitos:IEEE Computer Society Press, 2005:45-54.
[21] CHEN C M, XU L, LEE T, et al. A flow-guided file layout for out-of-core streamline computation[C]//2012 IEEE Pacific Visualization Symposium. Piscataway:IEEE, 2012:145-152.
[22] CHEN C M, NOUANESENGSY B, LEE T Y, et al. Flow-guided file layout for out-of-core pathline computation[C]//IEEE Symposium on Large Data Analysis and Visualization (LDAV). Piscataway:IEEE, 2012:109-112.
[23] SISNEROS R, JONES C, HUANG J, et al. A multi-level cache model for run-time optimization of remote visualization[J]. IEEE Transactions on Visualization and Computer Graphics, 2007, 13(5):991-1003.
[24] PETERKA T, ROSS R, NOUANESENGSY B, et al. A study of parallel particle tracing for steady-state and time-varying flow fields[C]//2011 IEEE International Parallel & Distributed Processing Symposium. Piscataway:IEEE, 2011:580-591.
[25] NOUANESENGSY B, LEE T Y, LU K W, et al. Parallel particle advection and FTLE computation for time-varying flow fields[C]//Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. Piscataway:IEEE, 2012:1-11.
[26] NOUANESENGSY B, LEE T Y, SHEN H W. Load-balanced parallel streamline generation on large scale vector fields[J]. IEEE Transactions on Visualization and Computer Graphics, 2011, 17(12):1785-1794.
[27] CHEN L, FUJISHIRO I. Optimizing parallel performance of streamline visualization for large distributed flow datasets[C]//2008 IEEE Pacific Visualization Symposium. Piscataway:IEEE, 2008:87-94.
[28] YU H F, WANG C L, MA K L. Parallel hierarchical visualization of large time-varying 3D vector fields[C]//Proceedings of the 2007 ACM/IEEE Conference on Supercomputing. New York:ACM Press, 2007:12.
[29] BERGER, BOKHARI. A partitioning strategy for nonuniform problems on multiprocessors[J]. IEEE Transactions on Computers, 1987, C-36(5):570-580.
[30] SIMON H D. Partitioning of unstructured problems for parallel processing[J]. Computing Systems in Engineering, 1991, 2(2-3):135-148.
[31] KARYPIS G, KUMAR V. Parallel multilevel k-way partitioning scheme for irregular graphs[C]//Proceedings of the 1996 ACM/IEEE Conference on Supercomputing (CDROM). New York:ACM Press, 1996:35.
[32] CATALYUREK U V, BOMAN E G, DEVINE K D, et al. Hypergraph-based dynamic load balancing for adaptive scientific computations[C]//2007 IEEE International Parallel and Distributed Processing Symposium. Piscataway:IEEE, 2007:1-11.
[33] GUO H Q, ZHANG J, LIU R C, et al. Advection-based sparse data management for visualizing unsteady flow[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(12):2555-2564.
[34] GERNDT A, HENTSCHEL B, WOLTER M, et al. VIRACOCHA:an efficient parallelization framework for large-scale CFD post-processing in virtual environments[C]//Proceedings of the 2004 ACM/IEEE Conference on Supercomputing. Piscataway:IEEE, 2004:50.
[35] RAFTERY A E. A model for high-order Markov chains[J]. Journal of the Royal Statistical Society:Series B (Methodological), 1985, 47(3):528-539.
[36] ZHANG J, GUO H Q, YUAN X R. Efficient unsteady flow visualization with high-order access dependencies[C]//2016 IEEE Pacific Visualization Symposium (PacificVis). Piscataway:IEEE, 2016:80-87.
[37] HONG F, ZHANG J, YUAN X R. Access pattern learning with long short-term memory for parallel particle tracing[C]//2018 IEEE Pacific Visualization Symposium (PacificVis). Piscataway:IEEE, 2018:76-85.
[38] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[39] ZHANG J, GUO H Q, YUAN X R, et al. Dynamic data repartitioning for load-balanced parallel particle tracing[C]//2018 IEEE Pacific Visualization Symposium (PacificVis). Piscataway:IEEE, 2018:86-95.
[40] KARYPIS G, KUMAR V. ParMETIS:Parallel graph partitioning and sparse matrix ordering library[R]. Minnesota:University of Minnesota, 1997.
[41] ZHANG J, YANG C H, LI Y D, et al. LBVis:interactive dynamic load balancing visualization for parallel particle tracing[C]//2020 IEEE Pacific Visualization Symposium (PacificVis). Piscataway:IEEE, 2020:91-95.

Options

文章导航

模态框（Modal）标题

摘要

本文引用格式

Abstract

参考文献