Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

preprint · arXiv (Cornell University) · 2026

CGF-Softmax: A Cumulant-Based Softmax Reformulation for Efficient Inference under Homomorphic Encryption

Hanjun Park, Byeong-Seo Min, Jiheon Woo, Min-Wook Jeong, Jongho Shin, Yongwoo Lee, Young-Sik Kim, Yongjune Kim

Cited by 3Open access

Abstract

Homomorphic encryption (HE) is a prominent framework for privacy-preserving machine learning, enabling inference directly on encrypted data. However, evaluating softmax, a core component of transformer architectures, remains particularly challenging in HE due to its multivariate structure, the large dynamic range induced by exponential functions, and the costly division operation. In this paper, we propose CGF-softmax, which reformulates the softmax denominator through the cumulant generating function (CGF). By eliminating both homomorphic division and explicit maximum subtraction, this reformulation substantially reduces multiplicative depth while preserving key properties of softmax. Extensive experiments on Vision Transformers and large language models show that CGF-softmax provides an efficient and accurate approximation of softmax in encrypted inference. In particular, it achieves inference accuracy close to that of high-depth exact methods, while requiring substantially lower computational cost through reduced multiplicative depth.

Cite this paper

APA 7

Park, H., Min, B.-S., Woo, J., Jeong, M.-W., Shin, J., Lee, Y., Kim, Y.-S., & Kim, Y. (2026). CGF-Softmax: A Cumulant-Based Softmax Reformulation for Efficient Inference under Homomorphic Encryption. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2602.01621

MLA 9

Park, Hanjun, et al. “CGF-Softmax: A Cumulant-Based Softmax Reformulation for Efficient Inference under Homomorphic Encryption.” arXiv (Cornell University), 2026, https://doi.org/10.48550/arxiv.2602.01621.

Chicago

Park, Hanjun, Byeong-Seo Min, Jiheon Woo, Min-Wook Jeong, Jongho Shin, Yongwoo Lee, Young-Sik Kim, and Yongjune Kim. 2026. “CGF-Softmax: A Cumulant-Based Softmax Reformulation for Efficient Inference under Homomorphic Encryption.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2602.01621.

Harvard

Park, H. et al. (2026) ‘CGF-Softmax: A Cumulant-Based Softmax Reformulation for Efficient Inference under Homomorphic Encryption’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.2602.01621.

BibTeX
@misc{park2026softmax,
  title = {{CGF-Softmax: A Cumulant-Based Softmax Reformulation for Efficient Inference under Homomorphic Encryption}},
  author = {Park, Hanjun and Min, Byeong-Seo and Woo, Jiheon and Jeong, Min-Wook and Shin, Jongho and Lee, Yongwoo and Kim, Young-Sik and Kim, Yongjune},
  journal = {arXiv (Cornell University)},
  year = {2026},
  doi = {10.48550/arxiv.2602.01621}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.

Related papers