LM4HPC: Towards Effective Language Model Application in High-Performance Computing

Chen, Le; Lin, Pei-Hung; Vanderbruggen, Tristan; Liao, Chunhua; Emani, Murali; de Supinski, Bronis

doi:10.1007/978-3-031-40744-4_2

Cited by 6 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nichols et al [22] refined the application of LLMs to enhance the generation of OpenMP pragmas in parallel algorithm implementations, encompassing MPI cases. In a related context, Chen et al [23] introduced LM4HPC, a framework designed for HPCspecific tasks within the realm of LLMs. They emphasized the scarcity of training and evaluation datasets in the field of high-performance computing (HPC).…”

Section: Introductionmentioning

confidence: 99%

GPT-Driven Source-to-Source Transformation for Generating Compilable Parallel CUDA Code for Nussinov’s Algorithm

Palkowski,

Gruzewski

2024

Electronics

View full text Add to dashboard Cite

Designing automatic optimizing compilers is an advanced engineering process requiring a great deal of expertise, programming, testing, and experimentation. Maintaining the approach and adapting it to evolving libraries and environments is a time-consuming effort. In recent years, OpenAI has presented the GPT model, which is designed for many fields like computer science, image processing, linguistics, and medicine. It also supports automatic programming and translation between programming languages, as well as human languages. This article will verify the usability of the commonly known LLM model, GPT, for the non-trivial NPDP Nussinov’s parallel algorithm code within the OpenMP standard to create a parallel equivalent of CUDA for NVIDIA graphics cards. The goal of this approach is to avoid creating any post-processing scripts and writing any lines of target code. To validate the output code, we compare the resulting arrays with the ones calculated by the optimized code for the CPU generated employing the polyhedral compilers. Finally, the code will be checked for scalability and performance. We will concentrate on assessing the capabilities of GPT, highlighting common challenges that can be refined during future learning processes. This will enhance code generation for various platforms by leveraging the outcomes from polyhedral optimizers.

show abstract

Section: Introductionmentioning

confidence: 99%

GPT-Driven Source-to-Source Transformation for Generating Compilable Parallel CUDA Code for Nussinov’s Algorithm

Palkowski,

Gruzewski

2024

Electronics

View full text Add to dashboard Cite

show abstract

Large language model evaluation for high‐performance computing software development

Godoy,

Valero‐Lara,

Teranishi

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

We apply AI‐assisted large language model (LLM) capabilities of GPT‐3 targeting high‐performance computing (HPC) kernels for (i) code generation, and (ii) auto‐parallelization of serial code in C ++, Fortran, Python and Julia. Our scope includes the following fundamental numerical kernels: AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG, and language/programming models: (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). Kernel implementations are generated using GitHub Copilot capabilities powered by the GPT‐based OpenAI Codex available in Visual Studio Code given simple <kernel> + <programming model> + <optional hints> prompt variants. To quantify and compare the generated results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. For auto‐parallelization, we use ChatGPT interactively giving simple prompts as in a dialogue with another human including simple “prompt engineering” follow ups. Results suggest that correct outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general‐purpose Python can benefit from adding language keywords, while Julia prompts perform acceptably well for its Threads and CUDA.jl programming models. We expect to provide an initial quantifiable point of reference for code generation in each programming model using a state‐of‐the‐art LLM. Overall, understanding the convergence of LLMs, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human‐computer interactions.

show abstract

Finding Equivalent OpenMP Fortran and C/C++ Code Snippets Using Large Language Models

Sekender,

Lin,

Liao

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

LM4HPC: Towards Effective Language Model Application in High-Performance Computing

Cited by 6 publications

References 13 publications

GPT-Driven Source-to-Source Transformation for Generating Compilable Parallel CUDA Code for Nussinov’s Algorithm

GPT-Driven Source-to-Source Transformation for Generating Compilable Parallel CUDA Code for Nussinov’s Algorithm

Large language model evaluation for high‐performance computing software development

Finding Equivalent OpenMP Fortran and C/C++ Code Snippets Using Large Language Models

Contact Info

Product

Resources

About