preprint · ArXiv.org · 2025

Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computation

Yaoyao Ding, Bohan Hou, Xiao Zhang, A. Binus Y. Lin, Tianqi Chen, Cody Yu Hao, Yida Wang, Gennady Pekhimenko

Cited by 0Open access

Abstract

Serving Large Language Models (LLMs) is critical for AI-powered applications, yet it demands substantial computational resources, particularly in memory bandwidth and computational throughput. Low-precision computation has emerged as a key technique to improve efficiency while reducing resource consumption. Existing approaches for generating low-precision kernels are limited to weight bit widths that are powers of two and suffer from suboptimal performance because of high-level GPU programming abstractions. These abstractions restrict critical optimizations, such as fine-grained register management and optimized memory access patterns, that are essential for efficient low-precision computations. In this paper, we introduce Tilus, a domain-specific language designed for General-Purpose GPU (GPGPU) computing that supports low-precision data types with arbitrary bit widths from 1 to 8 while maintaining GPU programmability. Tilus features a thread-block-level programming model, a hierarchical memory space, a novel algebraic layout system, and extensive support for diverse low-precision data types. Tilus programs are compiled into highly efficient GPU programs through automatic vectorization and instruction selection. Extensive experiments demonstrate that Tilus efficiently supports a full spectrum of low-precision data types, and outperforms state-of-the-art low-precision kernels. Compared to existing compilers such as Triton and Ladder, as well as hand-optimized kernels such as QuantLLM and Marlin, Tilus achieves performance improvements of: $1.75\times$, $2.61\times$, $1.29\times$ and $1.03\times$, respectively. We open-source Tilus at https://github.com/NVIDIA/tilus.

Cite this paper

APA 7

Ding, Y., Hou, B., Zhang, X., Lin, A. B. Y., Chen, T., Hao, C. Y., Wang, Y., & Pekhimenko, G. (2025). Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computation. ArXiv.org. https://doi.org/10.48550/arxiv.2504.12984

MLA 9

Ding, Yaoyao, et al. “Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computation.” ArXiv.org, 2025, https://doi.org/10.48550/arxiv.2504.12984.

Chicago

Ding, Yaoyao, Bohan Hou, Xiao Zhang, A. Binus Y. Lin, Tianqi Chen, Cody Yu Hao, Yida Wang, and Gennady Pekhimenko. 2025. “Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computation.” ArXiv.org. https://doi.org/10.48550/arxiv.2504.12984.

Harvard

Ding, Y. et al. (2025) ‘Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computation’, ArXiv.org. Available at: https://doi.org/10.48550/arxiv.2504.12984.

BibTeX

@misc{ding2025tilus,
  title = {{Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computation}},
  author = {Ding, Yaoyao and Hou, Bohan and Zhang, Xiao and Lin, A. Binus Y. and Chen, Tianqi and Hao, Cody Yu and Wang, Yida and Pekhimenko, Gennady},
  journal = {ArXiv.org},
  year = {2025},
  doi = {10.48550/arxiv.2504.12984}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.