Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “Adaptive optimization”

Adaptive Robust Optimization

Xu Andy Sun, Antonio J. Conejo · RePEc: Research Papers in Economics

Abstract In Chap. 2 , we have studied static robust optimization with the following robust inequality constraint f ( x , ξ ) ≤ 0 , ∀ ξ ∈ U , $$\\displaystyle \\begin{aligned} f(\\boldsymbol {x},\\boldsymbol {\\xi })\\le 0, \\quad \\forall \\boldsymbol {\\xi }\\in \\mathcal {U},{} \\end{aligned} $$ where U $$\\mathcal {U}$$ is an uncertainty set, ξ is the uncertain parameter taking values in U $$\\mathcal {U}$$ , and x is the decision. This robust model seeks one robust decision x that is feasible for all uncertainty realizations in U $$\\mathcal {U}$$ . This embodies the canonical meaning of “robustness” in decision making. We call such a robust decision a static policy. However, in many situations, static policy may not be the best choice. More specifically, (1) we may need to deal with equality constraints involving uncertain parameters, or (2) the uncertainty may have some dynamic structures that call for the decisions to adapt accordingly. Both situations can be illustrated by examples from electric power systems. For the first situation, we know that the electricity supply must be equal to the electricity demand all the time. In other words, there is an equality constraint between supply and demand. As the electricity demand, viewed as an uncertain parameter, fluctuates between peaks and troughs everyday, the generation decision must also follow the demand, i.e., must be a function of the demand. A static policy for electricity generation in this case would amount to a fixed supply for all varying levels of demand, which clearly would lead to violation of the balance between generation and consumption.

Cited by 0

Adaptive Noisy Optimization

Olivier Teytaud · 2014

Abstract. In this paper, adaptive noisy optimization on variants of the noisy sphere model is considered, i.e. optimization in which the same algorithm is able to adapt to several frameworks, including some for which no bound has never been derived. Incidentally, bounds derived by [16] for noise quickly decreasing to zero around the optimum are extended to the more general case of a positively lower-bounded noise thanks to a careful use of Bernstein bounds (using empirical estimates of the variance) instead of Chernoff-like variants. 1

Cited by 0

Adaptive Bilevel Optimization

Kimon Antonakopoulos, Shoham Sabach, Luca Viano et al. · ACM / IMS Journal of Data Science · 2025

We propose a new adaptive optimization algorithm based on mirror descent for a class of possibly non-convex smooth bilevel optimization problems. The bilevel optimization template is broadly applicable in machine learning as it features two coupled problems where the optimal solution set of an inner problem serves as a constraint set for the outer problem. Currently, available algorithms require knowledge of both inner and outer gradient Lipschitz constants, which are difficult to tune in practice. By using an “on the fly” accumulation strategy on gradient norms, our adaptive algorithm circumvents this difficulty, and to our knowledge, is the first adaptive algorithm for bilevel optimization. In the convex setting, we obtain a convergence rate of \(\mathcal {O}(1/T)\) in terms of the outer objective function, where T is the number of iterations. For non-convex outer objective functions, our algorithm achieves a best-iterate guarantee of \(\mathcal {O}(1/T)\) in terms of the squared norm of the gradient of the outer objective function. Additionally, we provide numerical evidence to support the theory in a reinforcement learning setting.

Cited by 0

Adaptive Stochastic Optimization

Frank E. Curtis, Katya Scheinberg · arXiv (Cornell University) · 2020

Optimization lies at the heart of machine learning and signal processing. Contemporary approaches based on the stochastic gradient method are non-adaptive in the sense that their implementation employs prescribed parameter values that need to be tuned for each application. This article summarizes recent research and motivates future work on adaptive stochastic optimization methods, which have the potential to offer significant computational savings when training large-scale systems.

Cited by 4Open access

Adaptive Traffic Optimization

Ciprian Dobre, Andrei Ichimescu, Valentin Cristea · 2012

Wireless technologies can help solve traffic congestions in urban environments, where road infrastructures develop slower than the sometimes exponential growth in the number of cars in traffic. We present a system for traffic control and congestion avoidance. The system monitors traffic using cars acting as data collectors. It aggregates the monitoring data into a traffic model, further used by cars to dynamically adjust their routes. We propose novel metrics for traffic, and present evaluation results that show the capabilities of the proposed congestion avoidance model to correct traffic conditions.

Cited by 1

Robust adaptive bayesian optimization

Shang,Hairong, Fystro,Gregory, Borland,Michael et al. · JACOW · 2023

Particle accelerators require continuous adjustment to maintain beam quality. Several machine learning (ML) approaches are being explored for this task. At the Advanced Photon Source (APS), we have recently proposed the adaptive Bayesian optimization (ABO) algorithm and have shown it to be effective experimentally in the APS injector complex. Further testing has suggested several improvements, on which we report here. We introduce dynamic kernel switching, deep kernel learning, and surrogate model prior means, resulting in improved robustness. We also extend our code with multi-dimensional time kernel support and predictive constraint avoidance to make it applicable to a wider range of systems. These changes also improve the general ABO performance, but more importantly expand ABO applicability to systems with rapid or unexpected changes in either optimization parameters or time properties. Notably, this allows for rapid and automated fallback to conservative parameters when optimizer confidence degrades, with alarms raised for further operator review. These features will permit further operational ML adoption at APS.

Cited by 2Open access

Adaptive Distributionally Robust Optimization

Dimitris Bertsimas, Melvyn Sim, Meilin Zhang · Management Science · 2018

We develop a modular and tractable framework for solving an adaptive distributionally robust linear optimization problem, where we minimize the worst-case expected cost over an ambiguity set of probability distributions. The adaptive distributionally robust optimization framework caters for dynamic decision making, where decisions adapt to the uncertain outcomes as they unfold in stages. For tractability considerations, we focus on a class of second-order conic (SOC) representable ambiguity set, though our results can easily be extended to more general conic representations. We show that the adaptive distributionally robust linear optimization problem can be formulated as a classical robust optimization problem. To obtain a tractable formulation, we approximate the adaptive distributionally robust optimization problem using linear decision rule (LDR) techniques. More interestingly, by incorporating the primary and auxiliary random variables of the lifted ambiguity set in the LDR approximation, we can significantly improve the solutions, and for a class of adaptive distributionally robust optimization problems, exact solutions can also be obtained. Using the new LDR approximation, we can transform the distributionally adaptive robust optimization problem to a classical robust optimization problem with an SOC representable uncertainty set. Finally, to demonstrate the potential for solving management decision problems, we develop an algebraic modeling package and illustrate how it can be used to facilitate modeling and obtain high-quality solutions for medical appointment scheduling and inventory management problems. The electronic companion is available at https://doi.org/10.1287/mnsc.2017.2952 . This paper was accepted by Noah Gans, optimization.

Cited by 321

Adaptive Recursive Query Optimization

Anna Herlihy, Guillaume Martres, Anastasia Ailamaki et al. · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · 2024

Performance-critical industrial applications, including large-scale program, network, and distributed system analyses, are increasingly reliant on recursive queries for data analysis. Yet traditional relational algebra-based query optimization techniques do not scale well to recursive query processing due to the iterative nature of query evaluation, where relation cardinalities can change unpredictably during the course of a single query execution. To avoid error-prone cardinality estimation, adaptive query processing techniques use runtime information to inform query optimization, but these systems are not optimized for the specific needs of recursive query processing. In this paper, we introduce Adaptive Metaprogramming, an innovative technique that shifts recursive query optimization and code generation from compile-time to runtime using principled metaprogramming, enabling dynamic optimization and re-optimization before and after query execution has begun. We present a custom join-ordering optimization applicable at multiple stages during query compilation and execution. Through Carac, a custom Datalog engine, we evaluate the optimization potential of Adaptive Metaprogramming and show unoptimized recursive query execution time can be improved by three orders of magnitude and hand-optimized queries by 6x.

Cited by 3Open access

Soft Adaptive Policy Optimization

Chang Gao, Chujie Zheng, Xiong-Hui Chen et al. · ArXiv.org · 2025

Reinforcement learning (RL) plays an increasingly important role in enhancing the reasoning capabilities of large language models (LLMs), yet stable and performant policy optimization remains challenging. Token-level importance ratios often exhibit high variance-a phenomenon exacerbated in Mixture-of-Experts models-leading to unstable updates. Existing group-based policy optimization methods, such as GSPO and GRPO, alleviate this problem via hard clipping, making it difficult to maintain both stability and effective learning. We propose Soft Adaptive Policy Optimization (SAPO), which replaces hard clipping with a smooth, temperature-controlled gate that adaptively attenuates off-policy updates while preserving useful learning signals. Compared with GSPO and GRPO, SAPO is both sequence-coherent and token-adaptive. Like GSPO, SAPO maintains sequence-level coherence, but its soft gating forms a continuous trust region that avoids the brittle hard clipping band used in GSPO. When a sequence contains a few highly off-policy tokens, GSPO suppresses all gradients for that sequence, whereas SAPO selectively down-weights only the offending tokens and preserves the learning signal from the near-on-policy ones, improving sample efficiency. Relative to GRPO, SAPO replaces hard token-level clipping with smooth, temperature-controlled scaling, enabling more informative and stable updates. Empirical results on mathematical reasoning benchmarks indicate that SAPO exhibits improved training stability and higher Pass@1 performance under comparable training budgets. Moreover, we employ SAPO to train the Qwen3-VL model series, demonstrating that SAPO yields consistent performance gains across diverse tasks and different model sizes. Overall, SAPO provides a more reliable, scalable, and effective optimization strategy for RL training of LLMs.

Cited by 0Open access

Adaptive Recursive Query Optimization

Anna Herlihy, Guillaume Martres, Anastasia Ailamaki et al. · arXiv (Cornell University) · 2023

Performance-critical industrial applications, including large-scale program, network, and distributed system analyses, are increasingly reliant on recursive queries for data analysis. Yet traditional relational algebra-based query optimization techniques do not scale well to recursive query processing due to the iterative nature of query evaluation, where relation cardinalities can change unpredictably during the course of a single query execution. To avoid error-prone cardinality estimation, adaptive query processing techniques use runtime information to inform query optimization, but these systems are not optimized for the specific needs of recursive query processing. In this paper, we introduce Adaptive Metaprogramming, an innovative technique that shifts recursive query optimization and code generation from compile-time to runtime using principled metaprogramming, enabling dynamic optimization and re-optimization before and after query execution has begun. We present a custom join-ordering optimization applicable at multiple stages during query compilation and execution. Through Carac, a custom Datalog engine, we evaluate the optimization potential of Adaptive Metaprogramming and show unoptimized recursive query execution time can be improved by three orders of magnitude and hand-optimized queries by 6x.

Cited by 0Open access

Adaptive Multiobjective Memetic Optimization

Hieu V. Dang, Witold Kinsner · International Journal of Cognitive Informatics and Natural Intelligence · 2016

Multiobjective memetic optimization algorithms (MMOAs) are recently applied to solve nonlinear optimization problems with conflicting objectives. An important issue in an MMOA is how to identify the relative best solutions to guide its adaptive processes. In this paper, the authors introduce a framework of adaptive multiobjective memetic optimization algorithms (AMMOA) with an information theoretic criterion for guiding the adaptive selection, clustering, local learning processes, and a robust stopping criterion of AMMOA. The implementation of AMMOA is applied to several benchmark test problems with remarkable results. The paper also presents the application of AMMOA in designing an optimal image watermarking to maximize the quality of the watermarked images and the robustness of the watermark.

Cited by 4

Adaptive Particle Swarm Optimization

Zhi‐Hui Zhan, Jun Zhang, Yun Li et al. · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · 2009

An adaptive particle swarm optimization (APSO) that features better search efficiency than classical particle swarm optimization (PSO) is presented. More importantly, it can perform a global search over the entire search space with faster convergence speed. The APSO consists of two main steps. First, by evaluating the population distribution and particle fitness, a real-time evolutionary state estimation procedure is performed to identify one of the following four defined evolutionary states, including exploration, exploitation, convergence, and jumping out in each generation. It enables the automatic control of inertia weight, acceleration coefficients, and other algorithmic parameters at run time to improve the search efficiency and convergence speed. Then, an elitist learning strategy is performed when the evolutionary state is classified as convergence state. The strategy will act on the globally best particle to jump out of the likely local optima. The APSO has comprehensively been evaluated on 12 unimodal and multimodal benchmark functions. The effects of parameter adaptation and elitist learning will be studied. Results show that APSO substantially enhances the performance of the PSO paradigm in terms of convergence speed, global optimality, solution accuracy, and algorithm reliability. As APSO introduces two new parameters to the PSO paradigm only, it does not introduce an additional design or implementation complexity.

Cited by 1,996

Adaptive Sequential Stochastic Optimization

Craig Wilson, Venugopal V. Veeravalli, Angelia Nedich · IEEE Transactions on Automatic Control · 2018

A framework is introduced for sequentially solving convex stochastic minimization problems, where the objective functions change slowly, in the sense that the distance between successive minimizers is bounded. The minimization problems are solved by sequentially applying a selected optimization algorithm, such as stochastic gradient descent, based on drawing a number of samples in order to carry the iterations. Two tracking criteria are introduced to evaluate approximate minimizer quality: one based on being accurate with respect to the mean trajectory, and the other based on being accurate in high probability. An estimate of a bound on the minimizers' change, combined with properties of the chosen optimization algorithm, is used to select the number of samples needed to meet the desired tracking criterion. A technique to estimate the change in minimizers is provided along with analysis to show that eventually the estimate upper bounds the change in minimizers. This estimate of the change in minimizers provides sample size selection rules that guarantee that the tracking criterion is met for sufficiently large number of time steps. Simulations are used to confirm that the estimation approach provides the desired tracking accuracy in practice, while being efficient in terms of number of samples used in each time step.

Cited by 3Open access