preprint · ArXiv.org · 2025

Sequential Policy Gradient for Adaptive Hyperparameter Optimization

Cited by 0Open access

Abstract

Reinforcement learning is essential for neural architecture search and hyperparameter optimization, but the conventional approaches impede widespread use due to prohibitive time and computational costs. Inspired by DeepSeek-V3 multi-token prediction architecture, we propose Sequential Policy Gradient modeling (SPG), a novel trajectory generation paradigm for lightweight online hyperparameter optimization. In contrast to conventional policy gradient methods, SPG extends the base model with temporary modules, enabling it to generate state-action (padded) trajectories in a single forward pass. Our experiments demonstrate that models gain performance when retrained with SPG on their original datasets and also outperform standard transfer fine-tuning. We evaluate on five datasets spanning computer vision (ImageNet, COCO), natural language processing (GLUE, SQuAD), and audio (SUPERB) to assess the industrial applicability of SPG. The proposed method demonstrates consistent improvements across widely adopted models, achieving performance gains of $+0.2\sim7\%$, with significantly low computational costs. Fully reproducible code and pre-trained models: https://huggingface.co/UniversalAlgorithmic/SPG.

Cite this paper

APA 7

Li, Z., Cheng, J., & Gu, H. (2025). Sequential Policy Gradient for Adaptive Hyperparameter Optimization. ArXiv.org. https://doi.org/10.48550/arxiv.2506.15051

MLA 9

Li, Zheng, et al. “Sequential Policy Gradient for Adaptive Hyperparameter Optimization.” ArXiv.org, 2025, https://doi.org/10.48550/arxiv.2506.15051.

Chicago

Li, Zheng, Jerry Cheng, and Huanying Gu. 2025. “Sequential Policy Gradient for Adaptive Hyperparameter Optimization.” ArXiv.org. https://doi.org/10.48550/arxiv.2506.15051.

Harvard

Li, Z., Cheng, J. and Gu, H. (2025) ‘Sequential Policy Gradient for Adaptive Hyperparameter Optimization’, ArXiv.org. Available at: https://doi.org/10.48550/arxiv.2506.15051.

BibTeX

@misc{li2025sequential,
  title = {{Sequential Policy Gradient for Adaptive Hyperparameter Optimization}},
  author = {Li, Zheng and Cheng, Jerry and Gu, Huanying},
  journal = {ArXiv.org},
  year = {2025},
  doi = {10.48550/arxiv.2506.15051}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.