article · ArXiv.org · 2026
Non-Stationary Functional Bilevel Optimization
Jason Bohne, Ieva Petrulionyte, Michael Arbel, Mairal, Julien, Paweł Polak
Cited by 0Open access
Abstract
Functional bilevel optimization (FBO) provides a powerful framework for hierarchical learning in function spaces, yet current methods are limited to static offline settings and perform suboptimally in online, non-stationary scenarios. We propose SmoothFBO, the first algorithm for non-stationary FBO with both theoretical guarantees and practical scalability. SmoothFBO introduces a time-smoothed stochastic hypergradient estimator that reduces variance through a window parameter, enabling stable outer-loop updates with sublinear regret. Importantly, the classical parametric bilevel case is a special reduction of our framework, making SmoothFBO a natural extension to online, non-stationary settings. Empirically, SmoothFBO consistently outperforms existing FBO methods in non-stationary hyperparameter optimization and model-based reinforcement learning, demonstrating its practical effectiveness. Together, these results establish SmoothFBO as a general, theoretically grounded, and practically viable foundation for bilevel optimization in online, non-stationary scenarios.
Cite this paper
Bohne, J., Petrulionyte, I., Arbel, M., Mairal, J., & Polak, P. (2026). Non-Stationary Functional Bilevel Optimization. ArXiv.org.
Bohne, Jason, et al. “Non-Stationary Functional Bilevel Optimization.” ArXiv.org, 2026.
Bohne, Jason, Ieva Petrulionyte, Michael Arbel, Julien Mairal, and Paweł Polak. 2026. “Non-Stationary Functional Bilevel Optimization.” ArXiv.org.
Bohne, J. et al. (2026) ‘Non-Stationary Functional Bilevel Optimization’, ArXiv.org.
@article{bohne2026stationary,
title = {{Non-Stationary Functional Bilevel Optimization}},
author = {Bohne, Jason and Petrulionyte, Ieva and Arbel, Michael and Mairal, Julien and Polak, Paweł},
journal = {ArXiv.org},
year = {2026}
}Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.