Weinan Gao, Zhong‐Ping Jiang · 2015
This paper addresses the optimal output regulation problem of linear systems with unknown system dynamics. The exogenous signal is presumed to be generated by a continuous-time linear exosystem. Firstly, we formulate the linear optimal output regulation problem (LOORP). Then, we give an offline solution of LOORP to design the optimal static state-feedback servoregulator by solving an algebraic Riccati equation (ARE) and a regulator equation. Instead of solving these two equations directly, by using state, input and exogenous signals collected online, we employ an approximate/adaptive dynamic programming (ADP) technique to seek online approximations of above equations whereby we get the approximated optimal servoregulator. Rigorous stability analysis shows that the closed-loop linear system is exponentially stable. Also, the output of the plant asymptotically tracks the given reference. Simulation results demonstrate the effectiveness of the proposed approach.
Cited by 21
Tao Bian, Zhong‐Ping Jiang · Elsevier eBooks · 2016
Cited by 3
Frank L. Lewis, Draguna Vrabie · IEEE Circuits and Systems Magazine · 2009
Living organisms learn by acting on their environment, observing the resulting reward stimulus, and adjusting their actions accordingly to improve the reward. This action-based or reinforcement learning can capture notions of optimal behavior occurring in natural systems. We describe mathematical formulations for reinforcement learning and a practical implementation method known as adaptive dynamic programming. These give us insight into the design of controllers for man-made engineered systems that both learn and exhibit optimal behavior.
Cited by 1,511
Yuanheng Zhu, Dongbin Zhao, Haibo He · IEEE Transactions on Systems Man and Cybernetics Systems · 2019
For systems that can only be locally stabilized, control laws and their effective regions are both important. In this paper, invariant policy iteration is proposed to solve the optimal control of discrete-time systems. At each iteration, a given policy is evaluated in its invariantly admissible region, and a new policy and a new region are updated for the next iteration. Theoretical analysis shows the method is regionally convergent to the optimal value and the optimal policy. Combined with sum-of-squares polynomials, the method is able to achieve the near-optimal control of a class of discrete-time systems. An invariant adaptive dynamic programming algorithm is developed to extend the method to scenarios where system dynamics is not available. Online data are utilized to learn the near-optimal policy and the invariantly admissible region. Simulated experiments verify the effectiveness of our method.
Cited by 43Open access
Yufeng Gao, Dongyu Li · 2023
Cited by 0
Yu Jiang, Zhong‐Ping Jiang · 2013
This paper studies the robust optimal control design for uncertain nonlinear systems from a perspective of robust adaptive dynamic programming (robust-ADP). The objective is to fill up a gap in the past literature of ADP where dynamic uncertainties, or unmodeled dynamics, are not addressed. A key strategy is to integrate tools from modern nonlinear control theory, such as the robust redesign and the backstepping techniques as well as the nonlinear small-gain theorem, with the theory of ADP. The proposed robust-ADP methodology can be viewed as a natural extension of ADP to uncertain nonlinear systems. A practical learning algorithm is developed in this paper, and has been applied to analyze a sensorimotor control problem.
Cited by 8
Ding Wang, Haiming Huang, Derong Liu et al. · IEEE Transactions on Systems Man and Cybernetics Systems · 2024
In this article, an evolution-guided adaptive dynamic programming (EGADP) algorithm is developed to address the optimal regulation problems for the nonlinear systems. In the traditional adaptive dynamic programming algorithms, policy improvement is typically reliant on the gradient information, according to the first order necessity condition. However, these methods encounter limitations when calculating the gradient information becomes infeasible or system dynamics is not differentiable. In response to this challenge, the evolutionary computation is harnessed by EGADP to search for a superior policy during policy improvement. Therefore, compared with the traditional methods, scenarios that gradient information is unavailable can effectively be handled by EGADP. Additionally, the convergence of the algorithm is proven to enhance the rigorousness of the developed method. Finally, the three simulation experiments with realistic physical backgrounds are conducted to comprehensively demonstrate the effectiveness of the established method from different perspectives.
Cited by 21
Ruifeng Li, Liqian Dou, Hui Yu et al. · 2024
In this study, an adaptive dynamic programming-based collaborative tracking control algorithm is developed for the formation of morphing unmanned aerial vehicles (UAVs). The multi-morphing UAVs system is divided into the position and attitude subsystems. For the position subsystem, a novel cost function fused with the impact of UAV morphing is introduced and the position controllers for morphing UAVs are developed using adaptive dynamic programming (ADP). To quantify the impact of morphing, a high-order finite-time observer is introduced which estimates the acceleration from aerodynamic force acting on the wings. For the attitude subsystem, an integral back-stepping controller is constructed. The stability of morphing UAV formation is theoretically proven with the Lyapunov function. Simulations involving comparisons with PID algorithm and sliding mode control (SMC) method have been conducted to validate its effectiveness and robustness.
Cited by 0
Weinan Gao, Tianyou Chai, Nan Qiao et al. · 2024
String stability can be employed to evaluate the formation stability of vehicular platoons by checking if the velocity in the upstream direction of the vehicle platoon decreases, which is closely related to the safety of platooning systems in practice. Traditional string stable controllers are usually based on the accurate knowledge of vehicle model, which is hard to implement in practice. In this paper, we have proposed a novel data-driven control approach based on adaptive dynamic programming, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{H}_{\infty}$</tex> control, and zero-sum differential games, which does not rely on the model knowledge. Moreover, the platoon in closed-loop with the proposed control policy can ensure the strictly string stability. Traffic simulation results demonstrate the effectiveness of the proposed approach.
Cited by 0
Fang Fang · 2004
Cited by 6
Jian Fu, Haibo He, Xinmin Zhou · IEEE Transactions on Neural Networks · 2011
Adaptive dynamic programming (ADP) is a promising research field for design of intelligent controllers, which can both learn on-the-fly and exhibit optimal behavior. Over the past decades, several generations of ADP design have been proposed in the literature, which have demonstrated many successful applications in various benchmarks and industrial applications. While many of the existing researches focus on multiple-inputs-single-output system with steepest descent search, in this paper we investigate a generalized multiple-input-multiple-output (GMIMO) ADP design for online learning and control, which is more applicable to a wide range of practical real-world applications. Furthermore, an improved weight-updating algorithm based on recursive Levenberg-Marquardt methods is presented and embodied in the GMIMO approach to improve its performance. Finally, we test the performance of this approach based on a practical complex system, namely, the learning and control of the tension and height of the looper system in a hot strip mill. Experimental results demonstrate that the proposed approach can achieve effective and robust performance.
Cited by 96Open access
Matthias C. M. Troffaes · 2015
Adaptive control is often subject to serious bias in the learning phase, simply because insufficient information is available in order to motivate the choice of a unique prior, and hence, a unique optimal feedback. Imprecise
Cited by 0
Yu Jiang, Zhong‐Ping Jiang · 2017
Many tasks that humans perform in our everyday life involve different sources of uncertainties. However, it is interesting and surprising to notice how the central nervous system (CNS) can gracefully coordinate our movements to deal with these uncertainties. This chapter studies sensorimotor control with static and dynamic uncertainties under the framework of robust adaptive dynamic programming (RADP). The linear version of RADP is extended for stochastic systems by taking into account signal-dependent noise, and the proposed method is applied to study the sensorimotor control problem with both static and dynamic uncertainties. Results presented in the chapter suggest that the CNS may use RADP-like learning strategy to coordinate movements and to achieve successful adaptation in the presence of static and/or dynamic uncertainties. In the absence of dynamic uncertainties, the learning strategy reduces to an ADP-like mechanism.
Cited by 0
Wei Zhou, Jianyang Shi, Gang Yin et al. · IEEE Access · 2020
Optimal control of aluminum electrolysis production process (AEPP) has long been a challenging industrial issue due to its inherent difficulty in establishing an accurate dynamic model. In this paper, a novel robust optimal control algorithm based on adaptive dynamic programming (ADP) is proposed for the AEPP, where the system subjects to input constraints. First, to establish an accurate dynamic model for the AEPP system, recursive neural network (RNN) is employed to reconstruct the system dynamic using the input-output production data. To ensure input constraints are not to exceed the bound of the actuator, the optimal control problem of the AEPP is formulated under a new nonquadratic form performance index function. Then, considering the perturbation of the AEPP, the robust control problem is effectively converted to the constrained optimal control problem via system transformation. Furthermore, a single critic network framework is developed to obtain the approximate solution of the Hamilton-Jacobi-Bellman (HJB) equation. Finally, the proposed ADP controller is applied to the AEPP system to validate the effectiveness and performance.
Cited by 8Open access
Hongyan Guo, Guangyao Li, Jun Liu et al. · 2023
This paper presents a path tracking control method for autonomous vehicles by utilizing Adaptive Dynamic Programming (ADP). The primary goal is to enhance the path tracking performance under various longitudinal speeds. The proposed approach incorporates the consideration of nonlinear tire characteristics and establishes a nonlinear vehicle system model based on the fundamental principles of vehicle dynamics. By formulating a performance index function that captures the correlation between the vehicle’s position and desired paths, the optimal control strategy is obtained using the Dual Heuristic Programming algorithm (DHP), known for its computational efficiency, within the ADP framework. The simulation results verify the effectiveness of the proposed path tracking control scheme, which can have better control performance under different vehicle speeds.
Cited by 0
Linghuan Kong, Wei He, Chenguang Yang et al. · IEEE Transactions on Neural Networks and Learning Systems · 2020
We aim at the optimization of the tracking control of a robot to improve the robustness, under the effect of unknown nonlinear perturbations. First, an auxiliary system is introduced, and optimal control of the auxiliary system can be seen as an approximate optimal control of the robot. Then, neural networks (NNs) are employed to approximate the solution of the Hamilton-Jacobi-Isaacs equation under the frame of adaptive dynamic programming. Next, based on the standard gradient attenuation algorithm and adaptive critic design, NNs are trained depending on the designed updating law with relaxing the requirement of initial stabilizing control. In light of the Lyapunov stability theory, all the error signals can be proved to be uniformly ultimately bounded. A series of simulation studies are carried out to show the effectiveness of the proposed control.
Cited by 90
Lu Dong, Jun Yan, Xin Yuan et al. · IEEE Transactions on Cybernetics · 2018
This paper presents a functional model predictive control (MPC) approach based on an adaptive dynamic programming (ADP) algorithm with the abilities of handling control constraints and disturbances for the optimal control of nonlinear discrete-time systems. In the proposed ADP-based nonlinear MPC (NMPC) structure, a neural-network-based identification is established first to reconstruct the unknown system dynamics. Then, the actor-critic scheme is adopted with a critic network to estimate the index performance function and an action network to approximate the optimal control input. Meanwhile, as the MPC strategy can effectively determine the current control by solving a finite horizon open-loop optimal control problem, in the proposed algorithm, the infinite horizon is decomposed into a series of finite horizons to obtain the optimal control. In each finite horizon, the finite ADP algorithm solves the optimal control problem subject to the terminal constraint, the control constraint, and the disturbance. The uniform ultimate boundedness of the closed-loop system is verified by the Lyapunov approach. Finally, the ADP-based NMPC is conducted on two different cases and the simulation results demonstrate the quick response and strong robustness of the proposed method.
Cited by 86Open access
Chenyong Guan, Yu Jiang · Complex & Intelligent Systems · 2021
Abstract This paper studies the online learning control of a truck-trailer parking problem via adaptive dynamic programming (ADP). The contribution is twofold. First, a novel ADP method is developed for systems with parametric nonlinearities. It learns the optimal control policy of the linearized system at the origin, while the learning process utilizes online measurements of the full system and is robust with respect to nonlinear disturbances. Second, a control strategy is formulated for a commonly seen truck-trailer parallel parking problem, and the proposed ADP method is integrated into the strategy to provide online learning capabilities and to handle uncertainties. A numerical simulation is conducted to demonstrate the effectiveness of the proposed methodology.
Cited by 4Open access
Guohan Tang, Ding Wang, Ao Liu et al. · Neurocomputing · 2025
Cited by 10
Ting Huang, Derong Liu · 2011
In this paper, we apply adaptive dynamic programming to the residential energy system control and management, with an emphasis on home battery use connected to power grids. The proposed scheme is built upon a self-learning architecture with only a single critic module instead of the action-critic dual module architecture. The novelty of the present scheme is its ability to improve the performance as it learns and gains more experience in real-time operations under uncertain changes of the environment. Simulation results demonstrate that the proposed scheme can achieve the minimum electricity cost for residential customers.
Cited by 52