Shared⁃memory parallelization technology of unstructured CFD solver for multi⁃core CPU/many⁃core GPU architecture

Jian ZHANG; Ruitian LI; Liang DENG; Zhe DAI; Jie LIU; Chuanfu XU

doi:10.7527/S1000-6893.2023.28888

ACTA AERONAUTICAET ASTRONAUTICA SINICA >

2024 , Vol. 45 >Issue 7: 128888 - 128888

DOI: https://doi.org/10.7527/S1000-6893.2023.28888

Fluid Mechanics and Flight Mechanics

Shared⁃memory parallelization technology of unstructured CFD solver for multi⁃core CPU/many⁃core GPU architecture

Jian ZHANG ,
Ruitian LI ,
Liang DENG ,
Zhe DAI ,
Jie LIU ,
Chuanfu XU

Expand

^1.National Key Laboratory of Parallel and Distributed Computing，National University of Defense Technology，Changsha 410073，China
^2.Computational Aerodynamic Institute，China Aerodynamic Research and Development Center，Mianyang 621000，China

E-mail： dengliang11@nudt.edu.cn

Received date: 2023-04-19

Revised date: 2023-05-14

Accepted date: 2023-05-26

Online published: 2023-05-29

Supported by

National Numerical Wind-tunnel （NNW） Project of China;Sichuan Science and Technology Program(2023YFG0152)

Fold

Abstract

Shared memory parallelization for unstructured CFD on modern high-performance computer architecture is the key to improve the efficiency of floating point computing and realizing large-scale fluid simulation application capabilities. However， due to problems such as the complex topological relationship， poor data locality， and data write conflict in unstructured CFD computing， parallelization of the traditional algorithms in shared memory to efficiently explore the hardware capabilities of multi-core CPUs/many-core GPUs has become a significant challenge. Starting from industrial-level unstructured CFD software， a variety of shared memory parallel algorithms are designed and implemented by deeply analyzing the computing behavior and memory access mode， and data locality optimization technologies such as grid reordering， loop fusion， and multi-level memory access are used to further improve performance. Specifically， a comprehensive study is conducted on two parallel modes， loop-based and task-based， for multi-core CPU architectures. An innovative reduction parallel strategy based on a multi-level memory access optimization method is proposed for the many-core GPU architecture. All the parallel methods and optimization techniques implemented are deeply analyzed and evaluated by the test cases of the M6 wing and CHN-T1 airplane. The results show that the parallel strategy of division and replication has the best performance on the CPU platform. Using Cuthill-McKee grid renumbering and loop fusion techniques to optimize memory access can improve performance by 10%， respectively. For GPU platforms， the proposed reduction strategy combined with multi-level memory access optimization has a significant acceleration effect. For the hot spot subroutine with data racing， the speed-up can be further improved by 3 times， and the overall speed-up can reach 127.

Key words： unstructured-grid; CFD; shared memory parallelization; GPU; memory access optimization

Cite this article

Jian ZHANG , Ruitian LI , Liang DENG , Zhe DAI , Jie LIU , Chuanfu XU . Shared⁃memory parallelization technology of unstructured CFD solver for multi⁃core CPU/many⁃core GPU architecture[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2024 , 45(7) : 128888 -128888 . DOI: 10.7527/S1000-6893.2023.28888

References

1	阎超. 航空CFD四十年的成就与困境［J］. 航空学报， 2022， 43（10）： 526490.
	YAN C. Achievements and predicaments of CFD in aeronautics in past forty years［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（10）： 526490 （in Chinese）.
2	张子佩，赵钟，陈坚强，等. 风雷软件LES开发设计与验证［J］. 航空学报， 2023， 44（6）： 127171.
	ZHANG Z P， ZHAO Z， CHEN J Q， et al. Development and verification of LES model in NNW-PHengLEI［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（6）： 127171 （in Chinese）.
3	KROLL N， ABU-ZURAYK M， DIMITROV D， et al. DLR Project Digital-X： Towards virtual aircraft design and flight testing based on high-fidelity methods［J］. CEAS Aeronautical Journal， 2016， 7（1）： 3-27.
4	刘朋欣，袁先旭，孙东，等. 高温化学非平衡湍流边界层直接数值模拟［J］. 航空学报， 2022， 43（1）： 124877.
	LIU P X， YUAN X X， SUN D， et al. Direct numerical simulation of high-temperature turbulent boundary layer with chemical nonequilibrium［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（1）： 124877 （in Chinese）.
5	张来平，邓小刚，何磊，等. E级计算给CFD带来的机遇与挑战［J］. 空气动力学学报， 2016， 34（4）： 405-417.
	ZHANG L P， DENG X G， HE L， et al. The opportunity and grand challenges in computational fluid dynamics by exascale computing［J］. Acta Aerodynamica Sinica， 2016， 34（4）： 405-417 （in Chinese）.
6	刘胜，卢凯，郭阳，等. 一种自主设计的面向E级高性能计算的异构融合加速器［J］. 计算机研究与发展， 2021， 58（6）： 1234-1237.
	LIU S， LU K， GUO Y， et al. A self-designed heterogeneous accelerator for exascale high performance computing［J］. Journal of Computer Research and Development， 2021， 58（6）： 1234-1237 （in Chinese）.
7	龚春叶，刘杰，包为民，等. 后摩尔时代国产高性能并行应用软件生态建设综述［J］. 系统仿真学报， 2022， 34（10）： 2107-2118.
	GONG C Y， LIU J， BAO W M， et al. Review on ecological construction of domestic high-performance parallel application software in post Moore era［J］. Journal of System Simulation， 2022， 34（10）： 2107-2118 （in Chinese）.
8	CARY A， CHAWNER J， DUQUE E， et al. Realizing the vision of CFD in 2030［J］. Computing in Science & Engineering， 2022， 24（1）： 64-70.
9	ECONOMON T D， MUDIGERE D， BANSAL G， et al. Performance optimizations for scalable implicit RANS calculations with SU2［J］. Computers & Fluids， 2016， 129： 146-158.
10	GARCIA-GASULLA M， HOUZEAUX G， FERRER R， et al. MPI+X： Task-based parallelisation and dynamic load balance of finite element assembly［J］. International Journal of Computational Fluid Dynamics， 2019， 33（3）： 115-136.
11	ATKINSON PATRICK R. Enabling task parallelism for many-core architectures［D］. Bristol： University of Bristol， 2021
12	陈坚强，吴晓军，张健，等. FlowStar：国家数值风洞（NNW）工程非结构通用CFD软件［J］. 航空学报， 2021， 42（9）： 625739.
	CHEN J Q， WU X J， ZHANG J， et al. FlowStar： General unstructured-grid CFD software for national numerical windtunnel（NNW） project［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（9）： 625739 （in Chinese）.
13	王年华，常兴华，赵钟，等. 非结构CFD软件MPI+OpenMP混合并行及超大规模非定常并行计算的应用［J］. 航空学报， 2020， 41（10）： 123859.
	WANG N H， CHANG X H， ZHAO Z， et al. Implementation of hybrid MPI+OpenMP parallelization on unstructured CFD solver and its applications in massive unsteady simulations［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（10）： 123859 （in Chinese）.
14	张曦，孙旭，郭晓虎，等. 面向GPU的非结构网格有限体积计算流体力学的图染色方法优化［J］. 国防科技大学学报， 2022， 44（5）： 24-34.
	ZHANG X， SUN X， GUO X H， et al. Optimizations of graph coloring method for unstructured finite volume computational fluid dynamics on GPU［J］. Journal of National University of Defense Technology， 2022， 44（5）： 24-34 （in Chinese）.
15	GOMES P， ECONOMON T D， PALACIOS R. Sustainable high-performance optimizations in SU2［C］∥ Proceedings of the AIAA Scitech 2021 Forum. Reston： AIAA， 2021.
16	FARHAN M A AL， KEYES D E. Optimizations of unstructured aerodynamics computations for many-core architectures［J］. IEEE Transactions on Parallel and Distributed Systems， 2018， 29（10）： 2317-2332.
17	FOURNIER Y， BONELLE J， VEZOLLE P， et al. Multiple threads and parallel challenges for large simulations to accelerate a general Navier?Stokes CFD code on massively parallel systems［J］. Concurrency and Computation： Practice and Experience， 2013， 25（6）： 843-861.
18	THéBAULT L， PETIT E， DINH Q. Scalable and efficient implementation of 3d unstructured meshes computation： A case study on matrix assembly［C］∥ Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. New York： ACM， 2015： 120-129.
19	STONE C P， WALDEN A， ZUBAIR M， et al. Accelerating unstructured-grid CFD algorithms on NVIDIA and AMD GPUs［C］∥ 2021 IEEE/ACM 11th Workshop on Irregular Applications： Architectures and Algorithms （IA3）. New York： IEEE Press， 2021： 19-26.
20	BALOGH G D， REGULY I Z， MUDALIGE G R. Comparison of parallelisation approaches， languages， and compilers for unstructured mesh algorithms on GPUs［C］∥ International Workshop on Performance Modeling， Benchmarking and Simulation of High Performance Computer Systems. Berlin： Springer， 2018： 22-43.
21	KARYPIS G， KUMAR V. A fast and high quality multilevel scheme for partitioning irregular graphs［J］. SIAM Journal on Scientific Computing， 1998， 20（1）： 359-392.
22	CUTHILL E， MCKEE J. Reducing the bandwidth of sparse symmetric matrices［C］∥ Proceedings of the 1969 24th national conference. New York： ACM， 1969： 157-172.
23	FOURNIER Y， BONELLE J， MOULINEC C， et al. Optimizing Code_Saturne computations on petascale systems［J］. Computers & Fluids， 2011， 45（1）： 103-108.
24	OLIKER L， LI X Y， HUSBANDS P， et al. Effects of ordering strategies and programming paradigms on sparse matrix computations ［J］. SIAM Review， 2002， 44（3）： 373-393.
25	L?HNER R. Cache-efficient renumbering for vectorization［J］. International Journal for Numerical Methods in Biomedical Engineering， 2010， 26（5）： 628-636.
26	SULYOK A A， BALOGH G D， REGULY I Z， et al. Locality optimized unstructured mesh algorithms on GPUs［J］. Journal of Parallel and Distributed Computing， 2019， 134（C）： 50-64.
27	余永刚，周铸，黄江涛，等. 单通道客机气动标模CHN-T1设计［J］. 空气动力学学报， 2018， 36（3）： 505-513.
	YU Y G， ZHOU Z， HUANG J T， et al. Aerodynamic design of a standard model CHN-T1 for single-aisle passenger aircraft［J］. Acta Aerodynamica Sinica， 2018， 36（3）： 505-513 （in Chinese）.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References