Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “Adaptive Dynamic Programming Control”

Adaptive Dynamic Programming Neural Networks for Control

Shuai Li, Long Jin, Mohammed Aquil Mirza · 2019

This chapter proposes a self-learning control method applicable to Euler-Lagrange systems. In contrast to existing work on intelligent control of Euler-Lagrange systems, the stability of the closed loop system with the proposed method is proven in theory. On the other hand, different from model-based design strategies, such as backstepping-based design and variable structure-based design, the proposed method does not require information on the model parameters and therefore is a model independent method. The chapter formulates the problem from an optimal control perspective. In this framework, the goal is to find the input sequence to minimize the cost function defined on infinite horizon under the constraint of the system dynamics. The solution is found by solving a Bellman equation according to the principle of optimality. The chapter utilizes an adaptive dynamic programming strategy to numerically solve the input sequence in real time.

Cited by 1

On-Line Adaptive Dynamic Programming for Feedback Control

X. Zhong · 2017

Stability analysis and controller design are among the most important issues in feedback control problems. Usually, controller design for linear system can be obtained by solving the Riccati equation. However, when comes to the nonlinear control problem, Riccati equation becomes the well-known Hamilton-Jacobi-Bellman (HJB) equation which is difficult to tackle directly. Fortunately, adaptive dynamic programming (ADP) has been widely recognized as one of the “core methodologies” to achieve optimal control in stochastic process in a general case to achieve brain-like intelligent control. Extensive efforts and promising results have been achieved over the past decades. The achievements cover a large variety of problems, including system stability, convergence analysis, controller design, optimal control, state prediction, etc. This dissertation investigates the on-line ADP techniques for the feedback control systems and provides novel methods to solve several existing problems in this field. Specifically, the improvement and the original contribution of this dissertation can be summarized from algorithms, architectures, and applications, respectively. From the algorithms, an event-triggered ADP method is provided by sampling the efficient states rather than the entire system states generated during the learning process. The designed control law is only updated according to the sampled states to reduce the computation cost. In order to guarantee the sampled states are efficient, the theoretical analysis is provided to generate an event threshold to make sure the stability of the system during the event triggered learning process. It is said that only when the difference between the sampled and the current states is larger than the threshold, the system samples the state from the environment and updates the control law according to the sample states. This idea is further developed in the partially observable environment and the event threshold is designed only based on the observed feedback. A neural-network-based observer is designed to recover the internal states from the partially observable ones. Both the observer and the control law are updated aperiodically based on the sampled system outputs. In this way, the computation and the transmission load can be significantly reduced. From the simulation results, it is shown that the event-triggered ADP method can achieve competitive performance at the same time. From the architectures, a new framework, named “goal representation adaptive dynamic programming (GrADP)”, is proposed and introduced in this dissertation. It is regarded as the foundation of building intelligent systems through internal reward learning, goal representation and state-action association. Unlike the traditional ADP design

Cited by 0Open access

On-Line Adaptive Dynamic Programming for Feedback Control

Zhong, Xiangnan · Journal of Media Literacy Education · 2017

Stability analysis and controller design are among the most important issues in feedback control problems. Usually, controller design for linear system can be obtained by solving the Riccati equation. However, when comes to the nonlinear control problem, Riccati equation becomes the well-known Hamilton-Jacobi-Bellman (HJB) equation which is difficult to tackle directly. Fortunately, adaptive dynamic programming (ADP) has been widely recognized as one of the “core methodologies” to achieve optimal control in stochastic process in a general case to achieve brain-like intelligent control. Extensive efforts and promising results have been achieved over the past decades. The achievements cover a large variety of problems, including system stability, convergence analysis, controller design, optimal control, state prediction, etc.\nThis dissertation investigates the on-line ADP techniques for the feedback control systems and provides novel methods to solve several existing problems in this field. Specifically, the improvement and the original contribution of this dissertation can be summarized from algorithms, architectures, and applications, respectively.\nFrom the algorithms, an event-triggered ADP method is provided by sampling the efficient states rather than the entire system states generated during the learning process. The designed control law is only updated according to the sampled states to reduce the computation cost. In order to guarantee the sampled states are efficient, the theoretical analysis is provided to generate an event threshold to make sure the stability of the system during the event triggered learning process. It is said that only when the difference between the sampled and the current states is larger than the threshold, the system samples the state from the environment and updates the control law according to the sample states. This idea is further developed in the partially observable environment and the event threshold is designed only based on the observed feedback. A neural-network-based observer is designed to recover the internal states from the partially observable ones. Both the observer and the control law are updated aperiodically based on the sampled system outputs. In this way, the computation and the transmission load can be significantly reduced. From the simulation results, it is shown that the event-triggered ADP method can achieve competitive performance at the same time.\nFrom the architectures, a new framework, named “goal representation adaptive dynamic programming (GrADP)”, is proposed and introduced in this dissertation. It is regarded as the foundation of building intelligent systems through internal reward learning, goal representation and state-action association. Unlike the traditional ADP design with an action network and a critic network, this new approach integrates an additional goal network, such that to build a general internal reinforcement signal. Unlike the traditional fixed or predefined reinforcement signal, this new design can adaptively update the internal reinforcement representation over time and thus facilitate the system’s learning and optimization to accomplish the ultimate goals. This dissertation for the first time provides the theoretical foundation of the GrADP design. It is shown that the designed internal reinforcement signal can give the agent more information by considering more distance lookahead, and therefore, this signal is more efficient.\nFrom the applications, this dissertation designs the ADP method for a class of Markov jump systems (MJSs) to find the optimal control law even though the system state keeps jumping among several subsystems. This dissertation also shows that the control law obtained from the learning process can quickly converge to the optimal solutions which verifies the effectiveness of the proposed method.

Cited by 0Open access

Optimal Event-Triggered Control Using Adaptive Dynamic Programming

S. Jagannathan, Vignesh Narayanan, Avimanyu Sahoo · 2024

Optimal Event-triggered Control using Adaptive Dynamic Programming discusses event triggered controller design which includes optimal control and event sampling design for linear and nonlinear dynamic systems including networked control systems (NCS) when the system dynamics are both known and uncertain. The NCS are a first step to realize cyber-physical systems (CPS) or industry 4.0 vision. The authors apply several powerful modern control techniques to the design of event-triggered controllers and derive event-trigger condition and demonstrate closed-loop stability. Detailed derivations, rigorous stability proofs, computer simulation examples, and downloadable MATLAB® codes are included for each case. The book begins by providing background on linear and nonlinear systems, NCS, networked imperfections, distributed systems, adaptive dynamic programming and optimal control, stability theory, and optimal adaptive event-triggered controller design in continuous-time and discrete-time for linear, nonlinear and distributed systems. It lays the foundation for reinforcement learning-based optimal adaptive controller use for infinite horizons. The text then: Introduces event triggered control of linear and nonlinear systems, describing the design of adaptive controllers for them Presents neural network-based optimal adaptive control and game theoretic formulation of linear and nonlinear systems enclosed by a communication network Addresses the stochastic optimal control of linear and nonlinear NCS by using neuro dynamic programming Explores optimal adaptive design for nonlinear two-player zero-sum games under communication constraints to solve optimal policy and event trigger condition Treats an event-sampled distributed linear and nonlinear systems to minimize transmission of state and control signals within the feedback loop via the communication network Covers several examples along the way and provides applications of event triggered control of robot manipulators, UAV and distributed joint optimal network scheduling and control design for wireless NCS/CPS in order to realize industry 4.0 vision An ideal textbook for senior undergraduate students, graduate students, university researchers, and practicing engineers, Optimal Event Triggered Control Design using Adaptive Dynamic Programming instills a solid understanding of neural network-based optimal controllers under event-sampling and how to build them so as to attain CPS or Industry 4.0 vision.

Cited by 1

Assistive Power Buffer Control via Adaptive Dynamic Programming

Paolo Roberto Massenio, David Naso, Frank L. Lewis et al. · IEEE Transactions on Energy Conversion · 2020

Power buffers are power electronic converters, with large capacitors, that decouple volatile loads and a low-inertia distribution network in a DC microgrid. In this work, a set of distributed optimal control policies enable power buffers to reciprocally assist each other during abrupt load changes. While the majority of existing control paradigms are localized, enabling communication among buffers extends their effective range of assistance and helps them minimize a shared objective in a cooperative fashion. The control law's weights surfaces are learned for a mesh of reference loads of each power buffer. Hamilton-Jacobi-Bellman equation is solved by a continuous-time adaptive dynamic programming (ADP) approach with off-policy learning to directly provide a feedback controller, instead of existing approaches that obtain open-loop policies via Pontryagin's minimum principle. This paper presents the first attempt in using ADP techniques for the control of power buffers that respects their original nonlinear dynamics, overcoming the limitations of previous approaches based on small-signal analysis. Compared to the current literature, the proposed approach provides trained controllers that are known a priori, avoiding player-by-player solutions or real-time optimization procedures that could degrade performances or become computationally intensive. Hardware-in-the-loop emulations of a low-voltage DC microgrid validates the proposed approach.

Cited by 24

Adaptive control of powered transfemoral prostheses based on adaptive dynamic programming

Yue Wen, Ming Liu, Jennie Si et al. · 2016

In this study, we developed and tested a novel adaptive controller for powered transfemoral prostheses. Adaptive dynamic programming (ADP) was implemented within the prosthesis control to complement the existing finite state impedance control (FS-IC) in a prototypic active-transfemoral prosthesis (ATP). The ADP controller interacts with the human user-prosthesis system, observes the prosthesis user's dynamic states during walking, and learns to personalize user performance properties via online adaptation to meet the individual user's objectives. The new ADP controller was preliminarily tested on one able-bodied subject walking on a treadmill. The test objective was for the user to approach normative knee kinematics by tuning the FS-IC impedance parameters via ADP. The results showed the ADP was able to adjust the prosthesis controller to generate the desired normative knee kinematics within 10 minutes. In the meantime, the FS-IC impedance parameters converged at the end of the adaptive tuning procedure while maintaining the desired human-prosthesis performance. This study demonstrated the feasibility of ADP for adaptive control of a powered lower limb prosthesis. Future research efforts will address several important issues in order to validate the system on amputees. To achieve this goal, human user-centered performance objective functions will be developed, tested, and used in this adaptive controller design.

Cited by 27

Adaptive dynamic programming for the optimal liver regeneration control

V. V. Karieva, S. V. L’vov · V N Karazin Kharkiv National University Ser Mathematics Applied Mathematics and Mechanics · 2024

Every living organism interacts with an environment and uses that interaction for an improvement of its own adaptability, and, as a result, one’s survival and overall development. The process of evolution shows us that different species change methods of interaction with an environment with passage of time, which leads to natural selection and survival of the most adaptive ones. This learning, which based on actions, or reinforcement learning may embrace the idea of optimal behavior occurring in environmental systems. We describe mathematical formulas for reinforcement learning and the practical integration method also known as adaptive dynamic programming. That gives us the overall concept of controllers for artificial biological systems that both learn and show the optimal behavior. This paper reviews the formulation of the upper optimality problem, for which the optimal regulation strategy is guaranteed to be better or equivalent to objective regulation rules that can be observed in natural biological systems. In cases of optimal reinforcement learning algorithms the learning process itself moves from the analysis of the item take on system dynamics to the much higher level. The object of interest now is not the details of the system dynamics, but the quantity efficiency index, which clearly represents how optimally the control system works. Such scheme of reinforcement learning is learning technique of optimal behavior in order to monitor the response to non-optimal control strategies. The purpose of this article is to show the possibility of using of reinforcement learning methods, the adaptive dynamic programming (ADP) in particular, to control biological systems using feedback. This article shows the on-line methods for solving the problem of searching the upper optimality estimate with adaptive dynamic programming.

Cited by 0Open access

Adaptive polyhedral meshing for approximate dynamic programming in control

Antonio Sala, Leopoldo Armesto · Engineering Applications of Artificial Intelligence · 2021

This work proposes a new criterion for adaptive meshing in polyhedral partitions which interpolate a value function in Approximate Dynamic Programming (ADP) in optimal control problems. The criterion adds new points to a simplicial mesh, based on: a user-defined initial condition probability density function which determines ‘influential’ regions of the state space, uncertainty (variance) propagation, and temporal-difference error. A collection of lemmas justifies the algorithmic proposal. Comparative analysis with other options in literature highlights the advantages of our proposal. The developed methods are applied to simulation examples and an experimental robotic setup.

Cited by 9Open access

Adaptive dynamic programming as a theory of sensorimotor control

Yu Jiang, Zhong‐Ping Jiang · 2012

This paper studies the control mechanism in human arm movements from a perspective of approximate/adaptive dynamic programming (ADP). The control scheme is developed by incorporating Itô calculus with the ADP method for continuous-time stochastic linear systems. An online learning technique is presented to find a robust optimal control policy without knowing the system dynamics. Finally, the proposed method is applied to a single-joint movement control problem and is validated by computer simulations.

Cited by 4

Approximated Adaptive Dynamic Programming Control of Axial-Piston Pump

Jordan Kralev, Alexander Mitov, Tsonyo Slavov · Mathematics · 2026

This article presents the synthesis, real-time implementation, and experimental validation of an approximated adaptive dynamic programming (AADP) actor–critic controller for precise flow rate regulation of a variable-displacement axial-piston pump designed for open-circuit hydraulic systems. Replacing the conventional hydro-mechanical regulator with an electrohydraulic proportional spool valve, the model-free controller employs two compact two-layer neural networks: the actor generates valve PWM signals from the flow tracking error, its integral, and measured discharge pressure, while the critic approximates the infinite-horizon quadratic cost-to-go via the online solution of the Bellman equation through gradient descent on Bellman residuals. Lyapunov analysis establishes closed-loop stability under bounded learning rates, with initial weights tuned via nominal plant simulation to ensure convergence from feasible starting policies. After extensive laboratory testing across four fixed loading conditions and dynamic load variations, the adaptive controller demonstrated superior performance compared with a proportional-integral (PI) controller, a Lyapunov model-reference adaptive controller (LMRAC), and an H∞ controller (Hinf). Real-time metrics confirm bounded critic signals and near-zero Bellman errors, validating optimal policy convergence amid unmodeled hydraulic nonlinearities.

Cited by 0Open access

Adaptive Dynamic Programming based Tracking Control for Hypersonic Vehicles

Dinghua Wang, Jie Wang, Gaowei Zhang et al. · 2023

In this paper, the control problem of hypersonic vehicle is investigated. A nonlinear optimal learning control approach based on adaptive dynamic programming (ADP) is proposed. The controller is composed of sliding mode control (SMC) and ADP control to track the desired velocity and altitude with the external disturbances. First, the hypersonic dynamic model is linearized and the sliding mode control law is designed. Then the ADP controller is used to provide the optimal feedback learning control law by adaptively adjusting the weights of the evaluation network. The stability analysis is carried out with the Lyapunov stability theory. Finally, the effectiveness of the proposed approach is verified by mathematical simulation results.

Cited by 0