Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 2,411 papers for “GPU kernels”

Multiple kernels optimization in GPU

Yu Xin Sun · DR-NTU (Nanyang Technological University) · 2012

This project is developed in the NVIDIA CUDA C/C++ environment which is provided. All the equipment and software stacks are provided by Parallel and Distributed Computing Center. The objective of this project is to find a way to split the GPU kernel and perform GPU kernel scheduling based on the importance of GPU kernels. \n \n \n \nGeneral purpose computing GPU (GPGPU) is burgeoning technique to enhance the computation of parallel programs. However, the non-preemptive nature of GPU kernel possesses great challenges for applying GPU computing to real-time application. In this project, we proposed a solution to split GPU kernels from the host code and also developed a model to schedule the GPU kernels with the kernel splitting techniques proposed. \n \n \n \nIn the first part of this project, we focus on how to split GPU kernels and make GPU kernels preemptive. Our design is to split GPU kernel into smaller size of sub kernels by reducing the grid of blocks of threads to execute it. Then this kernel is executed multiple times to have same functionality as the original kernel. In order to this, kernel invocation is introduced additional parameters by modifying the PTX code. \n \n \n \nThe second part of this project is to develop a model that can perform GPU kernel scheduling based on the priority assigned with the kernel splitting techniques developed in the first part. Our design is a server-clients model. In this model, clients initiate the kernel invocation, and pass control to server, then the server handle the actual invocation of GPU kernels. Server takes the overcharge the execution of all the kernels so it can schedule them based on priority.

Cited by 0

Automated testing for GPU kernels

Peng, Chao · ERA · 2021

Graphics Processing Units (GPUs) are massively parallel processors offering performance acceleration and energy efficiency unmatched by current processors (CPUs) in computers. These advantages along with recent advances in the programmability of GPUs have made them widely used in various general-purpose computing domains. However, this has also made testing GPU kernels critical to ensure that their behaviour meets the requirements of the design and specification. Despite the advances in programmability, GPU kernels are hard to code and analyse due to the high complexity of memory sharing patterns, striding patterns for memory accesses, implicit synchronisation, and combinatorial explosion of thread interleavings. Existing few techniques for testing GPU kernels use symbolic execution for test generation that incur a high overhead, have limited scalability and do not handle all data types. In this thesis, we present novel approaches to measure test effectiveness and generate tests automatically for GPU kernels. To achieve this, we address significant challenges related to the GPU execution and memory model, and the lack of customised thread scheduling and global synchronisation. We make the following contributions: First, we present a framework, CLTestCheck, for assessing the quality of test suites developed for GPU kernels. The framework can measure code coverage using three different coverage metrics that are inspired by faults found in real kernel code. Fault finding capability of the test suite is also measured by the framework to seed different types of faults in the kernel and reported in the form of mutation score, which is the ratio of the number of uncovered faults to the total number of seeded faults. Second, with the goal of being fast, effective and scalable, we propose a test generation technique, CLFuzz, for GPU kernels that combines mutation-based fuzzing for fast test generation and selective SMT solving to help cover unreachable branches by fuzzing. Fuzz testing for GPU kernels has not been explored previously. Our approach for fuzz testing randomly mutates input kernel argument values with the goal of increasing branch coverage and supports GPU-specific data types such as images. When fuzz testing is unable to increase branch coverage with random mutations, we gather path constraints for uncovered branch conditions, build additional constraints to represent the context of GPU execution such as number of threads and work-group size, and invoke the Z3 constraint solver to generate tests for them. Finally, to help uncover inter work-group data races and replay these bugs with fixed work-group schedules, we present a schedule amplifier, CLSchedule, that simulates multiple work-group schedules, with which to execute each of the generated tests. By reimplementing the OpenCL API, CLSchedule executes the kernel with a fixed work-group schedule rather than the default arbitrary schedule. It also executes the kernel directly, without requiring the developer to manually provide boilerplate host code. The outcome of our research can be summarised as follows: 1. CLTestCheck is applied to 82 publicly available GPU kernels from industry-standard benchmark suites along with their test suites. The experiment reveals that CLTestCheck is capable of automatically measuring the effectiveness of test suites, in terms of code coverage, faulting finding capability and revealing data races in real OpenCL kernels. 2. CLFuzz can automatically generate tests and achieve close to 100% coverage and mutation score for the majority of the data set of 217 GPU kernels collected from open-source projects and industry-standard benchmarks. 3. CLSchedule is capable of exploring the effect of work-group schedules on the 217 GPU kernels and uncovers data races in 21 of them. The techniques developed in this thesis demonstrate that we can measure the effectiveness of tests developed for GPU kernels with our coverage criteria and fault seeding methods. The result is useful in highlighting code portions that may need developers' further attention. Our automated test generation and work-group scheduling approaches are also fast, effective and scalable, with small overhead incurred (average of 0.8 seconds) and scalability to large kernels with complex data structures.

Cited by 0Open access

Automatic Horizontal Fusion for GPU Kernels

Ao Li, Bojian Zheng, Gennady Pekhimenko et al. · 2022

We present automatic horizontal fusion, a novel optimization technique that complements the standard kernel fusion techniques for GPU programs. Unlike the standard fusion, whose goal is to eliminate intermediate data round trips, our horizontal fusion technique aims to increase the thread-level parallelism to hide instruction latencies. We also present HFUSE, a new source to source CUDA compiler that implements automatic horizontal fusion. Our experimental results show that the horizontal fusion can speed up the running time by 2.5% 60.8%. Our results reveal that the horizontal fusion is especially beneficial for fusing kernels with instructions that require different kinds of GPU resources (e.g., a memory-intensive kernel and a compute-intensive kernel).

Cited by 39

A Hoare Logic for GPU Kernels

Kensuke Kojima, Atsushi Igarashi · ACM Transactions on Computational Logic · 2017

We study a Hoare Logic to reason about parallel programs executed on graphics processing units (GPUs), called GPU kernels. During the execution of GPU kernels, multiple threads execute in lockstep, that is, execute the same instruction simultaneously. When the control branches, the two branches are executed sequentially, but during the execution of each branch only those threads that take it are enabled; after the control converges, all the threads are enabled and again execute in lockstep. In this article, we first consider a semantics in which all threads execute in lockstep (this semantics simplifies the actual execution model of GPUs) and adapt Hoare Logic to this setting by augmenting the usual Hoare triples with an additional component representing the set of enabled threads. It is determined that the soundness and relative completeness of the logic do not hold for all programs; a difficulty arises from the fact that one thread can invalidate the loop termination condition of another thread through shared memory. We overcome this difficulty by identifying an appropriate class of programs for which the soundness and relative completeness hold. Additionally, we discuss thread interleaving, which is present in the actual execution of GPUs but not in the lockstep semantics mentioned above. We show that if a program is race free, then the lockstep and interleaving semantics produce the same result. This implies that our logic is sound and relatively complete for race-free programs, even if the thread interleaving is taken into account.

Cited by 10

Equivalence Checking of ML GPU Kernels

Kshitij Dubey, Benjamin Driscoll, Anjiang Wei et al. · ArXiv.org · 2025

With the rapid progress of deep learning and large language models (LLMs), companies now spend enormous sums executing GPU kernels. These kernels have, therefore, become prime targets for aggressive optimization. Recent efforts increasingly leverage LLMs to generate GPU kernels, but make no formal guarantees about the generated kernels. We present the first equivalence checker for GPU kernels and use it to formally verify the correctness of machine learning (ML) kernels optimized by hand, by LLMs, and by compilers. We show that our equivalence checker is sound and, for a well-defined class of GPU kernels which includes the programs of interest, complete. Our implementation, VOLTA, can verify ML computations such as convolutions, matrix multiplications, and various attention mechanisms.

Cited by 0Open access

Performance Degradation Analysis of GPU Kernels

Jinpeng Lv, Guodong Li, Alan Humphrey et al. · 2012

Hardware accelerators (currently Graphical Processing Units or GPUs) are an important component in many existing high-performance computing solutions [5]. Their growth in variety and usage is expected to skyrocket [1] due to many reasons. First, GPUs offer impressive energy efficiencies [3]. Second, when properly programmed, they yield impressive speedups by allowing programmers to model their computation around many fine-grained threads whose focus can be rapidly switched during memory stalls. Unfortunately, arranging for high memory access efficiency requires developed computational thinking to properly decompose a problem domain to gain this efficiency. Our work currently addresses the needs of the CUDA [5] approach to programming GPUs. Two important classes of such rules are bank conflict avoidance rules that pertain to CUDA shared memory and coalesced access rules that pertain to global memory. The former requires programmers to generate memory addresses from consecutive threads that fall within separate shared memory banks. The latter requires programmers to generate memory addresses that permit coalesced fetches from the global memory. In previous work [6], we had, to some extent addressed the former through SMTbased methods. Several other efforts also address bank conflicts [7, 8, 4]. In this work, we address the latter requirement—detecting when coalesced access rules are being violated.

Cited by 3

KernelBench: Can LLMs Write Efficient GPU Kernels?

Anne Ouyang, Simon Guo, Simran Arora et al. · ArXiv.org · 2025

Efficient GPU kernels are crucial for building performant machine learning architectures, but writing them is a time-consuming challenge that requires significant expertise; therefore, we explore using language models (LMs) to automate kernel generation. We introduce KernelBench, an open-source framework for evaluating LMs' ability to write fast and correct kernels on a suite of 250 carefully selected PyTorch ML workloads. KernelBench represents a real-world engineering environment and making progress on the introduced benchmark directly translates to faster practical kernels. We introduce a new evaluation metric fast_p, which measures the percentage of generated kernels that are functionally correct and offer a speedup greater than an adjustable threshold p over baseline. Our experiments across various state-of-the-art models and test-time methods show that frontier reasoning models perform the best out of the box but still fall short overall, matching the PyTorch baseline in less than 20% of the cases. While we show that results can improve by leveraging execution and profiling feedback during iterative refinement, KernelBench remains a challenging benchmark, with its difficulty increasing as we raise speedup threshold p.

Cited by 0Open access

Automatic Horizontal Fusion for GPU Kernels Artifact

Li, Ao, Bojian Zheng, Pekhimenko, Gennady et al. · Zenodo (CERN European Organization for Nuclear Research) · 2021

This artifact includes all code and scripts to reproduce the result of our paper: Automatic Horizontal Fusion for GPU Kernels.

Cited by 0Open access

Hok: Higher-Order GPU kernels in Elixir

André Rauber Du Bois, Tiago Perlin, Frederico Peixoto Antunes et al. · 2024

GPUs (Graphics Processing Units) are usually programmed using low-level languages like CUDA or OpenCL. Although these languages allow the implementation of very optimized software, they are difficult to program due to their low-level nature, where programmers have to mix coordination code, i.e., how tasks are created and distributed, with the actual computation code. In this paper we present Hok, an extension to the Elixir functional language that allows the implementation of higher-order GPU kernels, granting programmers the ability to clearly separate coordination from computation. The Hok system provides a DSL (Domain-Specific Language) for writing low-level GPU kernels that can be parameterized with the computation code. Hok allows device functions, including anonymous functions, to be created and referenced in the host code so that they can configure a kernel before it is launched.We demonstrate that Hok can be used to implement high-level abstractions such as algorithmic skeletons and array comprehensions. We also present experiments that demonstrate the usability of the current implementation of Hok, and show that high speedups can be obtained in comparison to pure Elixir, specially in computationally intensive programs with large inputs.

Cited by 1Open access

Experiments for Predictable Execution of GPU Kernels

Kreiliger F., Matějka J., Sojka M. et al. · Czech Technical University Digital Library (Czech Technical University in Prague) · 2019

Multi-Processor Systems-on-Chip (MPSoC) platforms will definitely power various future autonomous machines. Due to the high complexity of such platforms, it is difficult to achieve timing predictability, reliability and efficient resource utilization at the same time. We believe that time-triggered scheduling in combination with PRedictable Execution Model (PREM) can provide strong safety guarantees, and our longerterm goal is to schedule execution on the whole MPSoC (CPUs and GPU) in time triggered manner. To extend PREM to GPUs, we compare two synchronization mechanisms available on the NVIDIA Tegra X2 platform: one based on pinned memory and another that uses a GPU timer (socalled globaltimer). We found that running the NVIDIA profiler (nvprof) reconfigures the resolution of the globaltimer from 1 µs to 160 ns. By using time-triggered scheduling with such a resolution,itwaspossibletoreduceexecutiontimejitterofatiled 2D convolution kernel from 6.47% to 0.15% while maintaining the same average execution time.

Cited by 0Open access

A refactoring tool to extract GPU kernels

Kostadin Damevski, Madhan Muralimanohar · 2011

Significant performance gains can be achieved by using hardware architectures that integrate GPUs with conventional CPUs to form a hybrid and highly parallel computational engine. However, programming these novel architectures is tedious and error prone, reducing their ease of acceptance in an even wider range of computationally intensive applications. In this paper we discuss a refactoring technique, called Extract Kernel that transforms a loop written in C into a parallel function that uses NVIDIA's CUDA framework to execute on a GPU. The selected approach and the challenges encountered are described, as well as some early results that demonstrate the potential of this refactoring.

Cited by 8

On Static Timing Analysis of GPU Kernels

Vesa Hirvisalo · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2014

We study static timing analysis of programs running on GPU accelerators. Such programs follow a data parallel programming model that allows massive parallelism on manycore processors. Data parallel programming and GPUs as accelerators have received wide use during the recent years. The timing analysis of programs running on single core machines is well known and applied also in practice. However for multicore and manycore machines, timing analysis presents a significant but yet not properly solved problem. In this paper, we present static timing analysis of GPU kernels based on a method that we call abstract CTA simulation. Cooperative Thread Arrays (CTA) are the basic execution structure that GPU devices use in their operation that proceeds in thread groups called warps. Abstract CTA simulation is based on static analysis of thread divergence in warps and their abstract scheduling.

Cited by 6Open access

Automatic Horizontal Fusion for GPU Kernels Artifact

Li, Ao, Bojian Zheng, Pekhimenko, Gennady et al. · Zenodo (CERN European Organization for Nuclear Research) · 2021

This artifact includes all code and scripts to reproduce the result of our paper: Automatic Horizontal Fusion for GPU Kernels.

Cited by 0Open access

Securing GPU Kernels with Lightweight Formal Methods

Jiacheng Zhao, Baojian Hua · 2025

This paper reports our initial experience of applying lightweight formal method to validate the security of CUDA, the main programming language used for GPGPU programming. By “lightweight formal method”, we mean a pragmatic approach to verifying the security of CUDA programs coded by programmers. We do not aim to achieve full formal verification, but instead emphasize automation, usability, and the ability to continually ensure security as program evolve over time. Our approach first analyzes the program annotated with specification and loop invariants, and generates the corresponding verification conditions, which can be verified to ensure the security that is essential to GPU programs. Our work shows promising potentials by successfully verifying representative CUDA programs, including matrix multiplication optimized by using shared memory. Compared with some existing tools, our method exhibits better performance in CUDA programs security verification of without excessively increasing the programmer’s usage cost.

Cited by 0