Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “adaptive dynamic programming”

Adaptive Dynamic Programming

Jiayue Sun, Shun Xu, Yang Liu et al. · 2023

This open access book focuses on practical application of adaptive dynamic programming in chemotherapy drug delivery.

Cited by 4Open access

Adaptive dynamic programming

Song, Ruizhuo, Wei, Qinglai, Li, Qing · CERN Document Server (European Organization for Nuclear Research) · 2019

Cited by 0

An Adaptive Dynamic Programming Algorithm

For Stochastic Multiproduct, Katerina P. Papadaki, Warren B. Powell · 2002

We address the problem of dispatching a vehicle with di#erent product classes. There is a common dispatch cost, but holding costs that vary by product class. The problem exhibits multidimensional state, outcome and action spaces, and as a result is computationally intractable using either discrete dynamic programming methods, or even as a deterministic integer program. We prove a key structural property for the decision function, and exploit this property in the development of continuous value function approximations that form the basis of an approximate dispatch rule. Comparisons on single product-class problems, where optimal solutions are available, demonstrate solutions that are within a few percent of optimal. The algorithm is then applied to a problem with 100 product classes, and comparisons against a carefully tuned myopic heuristic demonstrate significant improvements.

Cited by 0

Particle SwarnOptimized Adaptive Dynamic Programming

Dongbin Zhao, Jianqiang Yi · 2007

Particle swarmoptimization isusedforthetraining oftheaction network andcritic network oftheadaptive dynamic programming approach. Thetypical structures oftheadaptive dynamicprogramming andparticle swarmoptimization are adopted forcomparison tootherlearning algorithms suchas gradient descent method. Besides simulation onthebalancing ofa cartpoleplant, amorecomplex plant pendulum robot (pendubot) istested forthelearning performance. Comparedtotraditional adaptive dynamicprogramming approaches, theproposed evolutionary learning strategy isverified asfaster convergence andhigher efficiency. Furthermore, thestructure becomes simple because theplant modeldoesnotneedtobeidentified beforehand.

Cited by 0

On Iterative Adaptive Dynamic Programming

Chang Qing Cui, Yi Qiang Wang, Chun Yang et al. · Applied Mechanics and Materials · 2013

Linear system is very seldom in actual control works, so there is more engineering significant of researching on the actual nonlinear system. Time delay phenomenon is the objective phenomenon exists in nature. Neural network based on adaptive dynamic programming principle is selected to implement algorithm. The algorithm contains model network training, H network training for time delay function, critic network training. Before running this iterative algorithm, training the model network first, the model uses a three-layer BP network to realize. Time delay function network H(K) is to approximate the functional relationship between the current control input and the delayed input. The critic network is used to approximate system performance function. The simulation results show that the proposed iterative adaptive dynamic programming can solve for the optimal control of delay nonlinear systems.

Cited by 0

The Adaptive Dynamic Programming Toolbox

Xiaowei Xing, Dong Eui Chang · Sensors · 2021

The paper develops the adaptive dynamic programming toolbox (ADPT), which is a MATLAB-based software package and computationally solves optimal control problems for continuous-time control-affine systems. The ADPT produces approximate optimal feedback controls by employing the adaptive dynamic programming technique and solving the Hamilton-Jacobi-Bellman equation approximately. A novel implementation method is derived to optimize the memory consumption by the ADPT throughout its execution. The ADPT supports two working modes: model-based mode and model-free mode. In the former mode, the ADPT computes optimal feedback controls provided the system dynamics. In the latter mode, optimal feedback controls are generated from the measurements of system trajectories, without the requirement of knowledge of the system model. Multiple setting options are provided in the ADPT, such that various customized circumstances can be accommodated. Compared to other popular software toolboxes for optimal control, the ADPT features computational precision and time efficiency, which is illustrated with its applications to a highly non-linear satellite attitude control problem.

Cited by 9Open access

A Dynamic Programming Approach to Adaptive

Jagdish Ramakrishnan, David Craft, Thomas Bortfeld et al. · 2016

Abstract. We conduct a theoretical study of various solution methods for the adaptive fractionation problem. The two messages of this paper are: (i) dynamic programming (DP) is a useful framework for adaptive radiation therapy, particularly adaptive fractionation, because it allows us to assess how close to optimal different methods are, and (ii) heuristic methods proposed in this paper are near-optimal, and therefore, can be used to evaluate the best possible benefit of using an adaptive fraction size. The essence of adaptive fractionation is to increase the fraction size when the tumor and organ-at-risk (OAR) are far apart (a “favorable ” anatomy) and to decrease the fraction size when they are close together. Given that a fixed prescribed dose must be delivered to the tumor over the course of the treatment, such an approach results in a lower cumulative dose to the OAR when compared to that resulting from standard fractionation. We first establish a benchmark by using the DP algorithm to solve the problem exactly. In this case, we characterize the structure of an optimal policy, which provides guidance for our choice of heuristics. We develop two intuitive, numerically near-optimal heuristic policies, which could be used for more complex, high-dimensional problems. Furthermore, one of the heuristics requires only a statistic of the motion probability distribution, making it a reasonable method for use in a realistic setting. Numerically, we find that the amount of decrease in dose to the OAR can vary significantly (5- 85%) depending on the amount of motion in the anatomy, the number of fractions, and the range of fraction sizes allowed. In general, the decrease in dose to the OAR is more pronounced when: (i) we have a high probability of large tumor-OAR distances, (ii) we use many fractions (as in a hyper-fractionated setting), and (iii) we allow large daily fraction size deviations. ar X iv

Cited by 0

Hamiltonian-Driven Hybrid Adaptive Dynamic Programming

Yongliang Yang, Kyriakos G. Vamvoudakis, Hamidreza Modares et al. · IEEE Transactions on Systems Man and Cybernetics Systems · 2020

This article presents a model-based hybrid adaptive dynamic programming (ADP) framework consisting of continuous feedback-based policy evaluation and policy improvement steps as well as an intermittent policy implementation procedure. This results in an intermittent ADP with a quantifiable performance and guaranteed closed-loop stability of the equilibrium point. To investigate the effect of aperiodic sampling on the communication bandwidth and the control performance of the intermittent ADP algorithms, we use a Hamiltonian-driven unified framework. With such a framework, it is shown that there is a tradeoff between the communication burden and the control performance. We finally show that the developed policies exhibit Zeno-free behaviors. Simulation examples show the efficiency of the proposed framework along with quantifiable comparisons of the policies with different intermittent information.

Cited by 80

Unit Commitment using Adaptive Dynamic Programming

J.A. Momoh, Yi Zhang · 2006

This paper solves a unit commitment (UC) problem of the interconnected power system. The UC provides an approach to decrease the cost and improve reliability of the network. The UC is a dynamic process, and the generation plan is always changing according to different load and network topology. The problem is solved by ADP which solves the dynamic problem with a two-stage neural network method. One stage is to search the optimal solution and the other stage is to evaluate the solution and obtain the objective function. Finally the UC of one sample power system is solved by ADP method

Cited by 20

Discrete-Time Impulsive Adaptive Dynamic Programming

Qinglai Wei, Ruizhuo Song, Zehua Liao et al. · IEEE Transactions on Cybernetics · 2020

In this paper, a new iterative adaptive dynamic programming (ADP) algorithm is developed to solve optimal impulsive control problems for infinite horizon discrete-time nonlinear systems. Considering the constraint of the impulsive interval, in each iteration, the iterative impulsive value function under each possible impulsive interval is obtained, and then the iterative value function and iterative control law are achieved. A new convergence analysis method is developed which proves an iterative value function to converge to the optimum as the iteration index increases to infinity. The properties of the iterative control law are analyzed, and the detailed implementation of the optimal impulsive control law is presented. Finally, two simulation examples with comparisons are given to show the effectiveness of the developed method.

Cited by 96

A dynamic programming approach to adaptive fractionation

Jagdish Ramakrishnan, David Craft, Thomas Bortfeld et al. · Physics in Medicine and Biology · 2012

We conduct a theoretical study of various solution methods for the adaptive fractionation problem. The two messages of this paper are as follows: (i) dynamic programming (DP) is a useful framework for adaptive radiation therapy, particularly adaptive fractionation, because it allows us to assess how close to optimal different methods are, and (ii) heuristic methods proposed in this paper are near-optimal, and therefore, can be used to evaluate the best possible benefit of using an adaptive fraction size. The essence of adaptive fractionation is to increase the fraction size when the tumor and organ-at-risk (OAR) are far apart (a 'favorable' anatomy) and to decrease the fraction size when they are close together. Given that a fixed prescribed dose must be delivered to the tumor over the course of the treatment, such an approach results in a lower cumulative dose to the OAR when compared to that resulting from standard fractionation. We first establish a benchmark by using the DP algorithm to solve the problem exactly. In this case, we characterize the structure of an optimal policy, which provides guidance for our choice of heuristics. We develop two intuitive, numerically near-optimal heuristic policies, which could be used for more complex, high-dimensional problems. Furthermore, one of the heuristics requires only a statistic of the motion probability distribution, making it a reasonable method for use in a realistic setting. Numerically, we find that the amount of decrease in dose to the OAR can vary significantly (5-85%) depending on the amount of motion in the anatomy, the number of fractions and the range of fraction sizes allowed. In general, the decrease in dose to the OAR is more pronounced when: (i) we have a high probability of large tumor-OAR distances, (ii) we use many fractions (as in a hyper-fractionated setting) and (iii) we allow large daily fraction size deviations.

Cited by 9Open access

Adaptive dynamic programming for linear impulse systems

Xiaohua Wang, Juan-juan Yu, Yao Huang et al. · Journal of Zhejiang University SCIENCE C · 2014

We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming (ADP) method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states, the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible, the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pre-impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training, optimal impulse control can be derived. Simulations are presented for illustrative purposes.

Cited by 4

Adaptive Dynamic Programming Based on Hierarchical Learning

巧 林 · Artificial Intelligence and Robotics Research · 2017

本文基于婴儿的认知发育模型LOC (Levels of Consciousness)提出了基于分层学习的自适应动态规划方法以改进学习和优化。根据LOC模型中感知的层次性以及工作目标的层次定义，为自适应动态规划设计了多层的目标网络结构及相应的分层学习方法。在自适应评价中引入多层的目标表征将引导系统做出好的决策并最终实现目标。文中给出了分层自适应动态规划的系统结构、学习和自适应过程，并通过模拟系统GLD (Green Light Domain)，在自适应交通信号控制模拟实验上验证了该方法的学习和控制能力。 This paper introduces an adaptive dynamic program method based on hierarchical learning. The motivations for this idea come from the levels of consciousness (LOC) model, which address the interdependence between consciousness and action in baby’s development. The introduction of a multilevel goal representation into the adaptive critic is able to guide the system’s decision-making to accomplish the long-term goal over time, mimicking certain levels of brain-like intelligence. The detailed system architecture, learning and adaption procedure are presented, and the learning and control capability of this approach is verified through light control in GLD (Green Light Domain).

Cited by 0

Adaptive dynamic programming for nonlinear impulse systems

Xiaohua Wang, Yanru Lian, Zhonghua Miao · 2017

Impulse systems describe impulsive phenomena and consider instantaneous impact on the states of the system. Optimal control of nonlinear impulse system hardly obtains analytical solutions considering the impulse nature. A new numerical algorithm based Adaptive dynamic programming (ADP) herein provides numerical approximation to the optimal control solutions. The difficulty designing ADP algorithm in such system lies in the policy update part. Variational method is adopted for policy update between two consecutive impulses. Simulations are presented for algorithm validations.

Cited by 0

Constrained Adaptive Dynamic Programming for Pid Controllers

Timotei Lala · 2025

In this paper, a model-free off-policy approximate constrained adaptive dynamic programming (ACADP) algorithm is developed for discrete-time general nonlinear systems. By using an adaptive learning rate, the controller update is constrained in a well-defined neighborhood to avoid performance degradation in case of high convergence uncertainty of the ACADP mechanism. The learning mechanism is used to tune a Proportional-Integrative-Derivative (PID) controller parameters in a model-free offline fashion. The Markovian assumption is proved for the closed-loop system with a general PID controller to show its applicability in Adaptive Dynamic Programming (ADP) like algorithms. The validation performed on a nonlinear process highlighting the stabilitypreserving attributes of the proposed method using the constrain mechanism.

Cited by 0

Intelligent Controllers Based on Adaptive Dynamic Programming

Vladimir Đorđević · SCIDAR · 2024

Određivanje tačnog modela je veoma izazovan i dugotrajan posao u većini praktičnih slučajeva, što algoritme upravljanja koji su zasnovani na modelu čini neizvodljivim. Čak i ako se može dobiti aproksimirani model sistema, dinamička neodređenost, prouzrokovana neslaganjem aproksimacije i realnog sistema, će degradirati performanse upravljanja klasično projektovanih regulatora. Stoga je projektovanje inteligentnih regulatora vođenih podacima, koji su robusni, adaptivni i pouzdani u prisustvu razčičitih neodređenosti i nepoznate dinamike, od primarnog značaja. U ovoj tezi se najpre predlažu inteligentni regulatori zasnovani na ADP-u linearnih sistema. Kod klasičnih algoritama upravljanja zasnovanih na ADP-u, uobičajeno se pretpostavlja da su promenljive veličine stanja merljive. Međutim, ova pretpostavka je generalno restriktivna, jer u mnogim inženjerskim primenama često nedostaje potpuna informacija o stanju, i samo je izlaz dostupan za merenje. U uslovima nepoznate dinamike sistema i nemerljivih stanja, od velikog interesa je korišćenje tehnika upravljanja zasnovanih na ulazno/izlaznim merenjima, koje se još nazivaju i tehnike sa povratnom spregom po izlazu. Takođe, stabilnost dinamičkih sistema se postiže putem povratne sprege po izlazu, pri čemu se za rekonstrukciju nepoznatih veličina stanja koriste ulazna i izlazna merenja. Da bi se ispunili zahtevi perzistente pobude, potrebno je dodati neki istraživački šum upravljačkom ulazu. Primenom teorije planiranja eksperimenta, u tezi se primenjuje optimalni istraživački šum u vidu sume sinusoida, koji će omogućiti da izlaz sistema nosi maksimum informacije o sistemu. Time će biti skraćeno vreme učenja, odnosno ubrzan proces projektovanja inteligentnih regulatora. Implementacijom strategije upravljanja koji su pokrenuti događajima smanjuje se broj uključivanja predloženih inteligentnih regulatora, čime se štede energetski, računski i komunikacioni resursi. Predložena strategija upravljanja će unaprediti algoritme upravljanja tolerantne na otkaze, čime su poboljšane performanse sistema upravljanja u slučaju pojave otkaza. Praktična primenljivost predloženih inteligentnih regulatora je povećana činjenicom da se nelinearni sistemi mogu verno predstaviti linearnim modelima sa estimacijom dinamike u realnom vremenu. Projektovani regulatori su sposobni da upravljaju linearnim i nelinearnim sistemima sa potpuno nepoznatim matricama sistema, a eksponencijalna konvergencija se postiže dok je uslov perzistentne pobude zadovoljen. Pored teorijskih okvira, praktični aspekt dobijenih rezultata je verifikovan na komleksnim inženjerskim problemima, kao što je upravljanje hidrauličkog servo aktuatora i samobalansirajućeg robota. Validnost i efektivnost predloženih inteligentih regulatora zasnovanih na ADP-u se evaluira kroz intenzivne simulacije.

Cited by 0Open access