preprint · arXiv (Cornell University) · 2025

Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective

Jiang, Jiacheng, Meng, Yuan, Tang, Chen, Yu, Han, Li, Qun, Wang, Zhi, Zhu, Wenwu

Cited by 0Open access

Abstract

Current quantization-aware training (QAT) methods primarily focus on enhancing the performance of quantized models on in-distribution (I.D) data, while overlooking the potential performance degradation on out-of-distribution (OOD) data. In this paper, we first substantiate this problem through rigorous experiment, showing that QAT can lead to a significant OOD generalization performance degradation. Further, we find the contradiction between the perspective that flatness of loss landscape gives rise to superior OOD generalization and the phenomenon that QAT lead to a sharp loss landscape, can cause the above problem. Therefore, we propose a flatness-oriented QAT method, FQAT, to achieve generalizable QAT. Specifically, i) FQAT introduces a layer-wise freezing mechanism to mitigate the gradient conflict issue between dual optimization objectives (i.e., vanilla QAT and flatness). ii) FQAT proposes an disorder-guided adaptive freezing algorithm to dynamically determines which layers to freeze at each training step, effectively addressing the challenges caused by interference between layers. A gradient disorder metric is designed to help the algorithm identify unstable layers during training. Extensive experiments on influential OOD benchmark demonstrate the superiority of our method over state-of-the-art baselines under both I.D and OOD image classification tasks.

Cite this paper

APA 7

Jiang, J., Meng, Y., Tang, C., Yu, H., Li, Q., Wang, Z., & Zhu, W. (2025). Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2509.00859

MLA 9

Jiang, Jiacheng, et al. “Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective.” arXiv (Cornell University), 2025, https://doi.org/10.48550/arxiv.2509.00859.

Chicago

Jiang, Jiacheng, Yuan Meng, Chen Tang, Han Yu, Qun Li, Zhi Wang, and Wenwu Zhu. 2025. “Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2509.00859.

Harvard

Jiang, J. et al. (2025) ‘Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.2509.00859.

BibTeX

@misc{jiang2025quantization,
  title = {{Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective}},
  author = {Jiang, Jiacheng and Meng, Yuan and Tang, Chen and Yu, Han and Li, Qun and Wang, Zhi and Zhu, Wenwu},
  journal = {arXiv (Cornell University)},
  year = {2025},
  doi = {10.48550/arxiv.2509.00859}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.