Thibault Langloisyz · 2007
In this paper we present a Reinforcement Learning method --- B-Learning --- for the control of a water production plant. A comparison between B-Learning and Dynamic Programming is provided from both theoretical and performance points of view. It is shown that Reinforcement-based neural control can lead to results comparable in quality to Dynamic Programming-based though less computationnally expensive. 1 Introduction Reinforcement Learning is an unsupervised method mainly used in the control area. It is based on a binary signal r(t) 1 called external reinforcement provided by the environment. This signal indicates whether the system is in an allowed state or not. Two functions have to be modeled for the construction of the controller. An evaluation function which produces an estimate of a long-term evaluation of the system state and a control function which provides the actions. Barto et al. [3] has introduced this learning method (Adaptive Heuristic Critic) for the control of an in...
Cited by 0
Daniel Jiang, Warren B. Powell · arXiv (Cornell University) · 2014
Many sequential decision problems can be formulated as Markov Decision\nProcesses (MDPs) where the optimal value function (or cost-to-go function) can\nbe shown to satisfy a monotone structure in some or all of its dimensions. When\nthe state space becomes large, traditional techniques, such as the backward\ndynamic programming algorithm (i.e., backward induction or value iteration),\nmay no longer be effective in finding a solution within a reasonable time\nframe, and thus we are forced to consider other approaches, such as approximate\ndynamic programming (ADP). We propose a provably convergent ADP algorithm\ncalled Monotone-ADP that exploits the monotonicity of the value functions in\norder to increase the rate of convergence. In this paper, we describe a general\nfinite-horizon problem setting where the optimal value function is monotone,\npresent a convergence proof for Monotone-ADP under various technical\nassumptions, and show numerical results for three application domains: optimal\nstopping, energy storage/allocation, and glycemic control for diabetes\npatients. The empirical results indicate that by taking advantage of\nmonotonicity, we can attain high quality solutions within a relatively small\nnumber of iterations, using up to two orders of magnitude less computation than\nis needed to compute the optimal solution exactly.\n
Cited by 0Open access
M. Priyanka · International Journal on Recent and Innovation Trends in Computing and Communication · 2015
Stereo matching algorithms are one of heavily researched topic in binocular stereo vision. Massive 3D information can be obtained by finding correct correspondence of different points between images captured from different views. Development of stereo matching algorithm is done for obtaining disparity maps i.e. depth information. When disparities computed for scan lines then dense reconstruction becomes time consuming for vision navigation systems. So for pair of stereo images proposed method extracts features points those are at contours in images and then a dynamic program is used to find the corresponding points from each image and calculates disparities. Also to reduce the noise which may lead to incorrect results in stereo correspondence, a new stereo matching algorithm based on the dynamic programming is proposed. Generally dynamic programming finds the global minimum for independent scan lines in polynomial time. While efficient, its performance is far from desired one because vertical consistency between scan lines is not enforced. This method review the use of dynamic programming for stereo correspondence by applying it to a contour instead to individual scan lines. Proposed methodology will obtain the global minimum for contour array in linear time using Longest Common Subsequent (LCS) dynamic programming method with no disparity space image (DSI). KeywordsStereo Correspondence, Contour, Dynamic Programming, Disparity space Image, Longest common subsequent. ________________________________________________________*****_______________________________________________________
Cited by 0Open access
Etienne Pellegrini, Ryan P. Russell · Acta Astronautica · 2020
Cited by 5
Guangming Tan, Ninghui Sun, Guang R. Gao · 2007
Dynamic programming is an efficient technique to solve combinatorial search and optimization problem. There have been many parallel dynamic programming algorithms. The purpose of this paper is to study a family of dynamic programming algorithm where data dependence appear between non-consecutive stages, in other words, the data dependence is non-uniform. This kind of dynnamic programming is typically called nonserial polyadic dynamic programming. Owing to the non-uniform data dependence, it is harder to optimize this problem for parallelism and locality on parallel architectures. In this paper, we address the chanllenge of exploiting fine grain parallelism and locality of nonserial polyadic dynamic programming on a multi-core architecture. We present a programming and execution model for multi-core architectures with memory hierarchy. In the framework of the new model, the parallelism and locality benifit from a data dependence transformation. We propose a parallel pipelined algorithm for filling the dynamic programming matrix by decomposing the computation operators. The new parallel algorithm tolerates the memory access latency using multi-thread and is easily improved with tile technique. We formulate and analytically solve the optimization problem determing the tile size that minimizes the total execution time. The experiments on a simulator give a validation of the proposed model and show that the fine grain parallel algorithm achieves sub-linear speedup and that a potential high scalability on multi-core arichitecture.
Cited by 64
Ye Chen, Shufan Wu, Xiaoliang Wang et al. · 2023
In this paper, a computational intelligent trajectory generation algorithm is proposed based on adaptive dynamic programming. The terminal homing guidance problem is first transformed to a nonlinear differential game problem, since it is hard to obtain an analytical solution for the Hamilton-Jacobi-Bellman (HJB) equation, an online guidance scheme is established by utilizing the policy iteration (PI) algorithm. The two procedures of evaluation and improvement are implemented by constructing actor-critic neural network (NN), and the approximate optimal solution can be obtained iteratively. On account of the weights of given network are updated by online data sampling, explicit information of the system internal dynamics is not needed. Finally, by applying this method to the terminal guidance scenario, numerical simulations demonstrate the effectiveness of the proposed law.
Cited by 1
Robert Willis, LeDon C. Jones · Journal of the Geological Society of India · 1987
Abstract Groundwater optimization models are predicated on the hydraulic or water quality response equations of the aquifer system. These equations relate the state variables of the groundwater system, the head or mass concentrations, and the decision variables that control the magnitude, location, and timing of pumping, artificial recharge, or waste injection. These management models are large-scale, non-convex programing problems. The objective of this paper is to present a differential dynamic programming (DDP) algorithm for solution of groundwater optimization problems. The effects of state and control variable constraints are discussed. The Jacobian matrices of the algorithm are evaluated analytically using a fully implicit, finite difference model of the aquifer system. DDP is used to identify optimal groundwater pumping schedules for a two-dimensional, unconfined aquifer system. The CPU time and central memory requirements of the algorithm are also presented.
Cited by 2
Guangming Tan, Shengzhong Feng, Ninghui Sun · 2006
Dynamic programming has been one of the most efficient approaches to sequence analysis and structure prediction in biology. However, their performance is limited due to the drastic increase in both the number of biological data and variety of the computer architectures. With regard to such predicament, this paper creates excellent algorithms aimed at addressing the challenges of improving memory efficiency and network latency tolerance for nonserial polyadic dynamic programming where the dependences are nonuniform. By relaxing the nonuniform dependences, we proposed a new cache oblivious scheme to enhance its performance on memory hierarchy architectures. Moreover we develop and extend a tiling technique to parallelize this nonserial polyadic dynamic programming using an alternate block-cyclic mapping strategy for balancing the computational and memory load, where an analytical parameterized model is formulated to determine the tile volume size that minimizes the total execution time and an algorithmic transformation is used to schedule the tile to overlap communication with computation to further minimize communication overhead on parallel architectures. The numerical experiments were carried out on several high performance computer systems. The new cache-oblivious dynamic programming algorithm achieve 2-10 speedup and the parallel tiling algorithm with communication-computation overlapping shows a desired potential for fine-grained parallel computing on massively parallel computer systems
Cited by 21
Tao Wang, Lu Xianliang, Mengshu Hou · Journal of Systems Engineering and Electronics · 2006
Replicas can improve the data reliability in distributed system. However, the traditional algorithms for replica management are based on the assumption that all replicas have the uniform reliability, which is inaccurate in some actual systems. To address such problem, a novel algorithm is proposed based on dynamic programming to manage the number and distribution of replicas in different nodes. By using Markov model, replicas management is organized as a multi-phase process, and the recursion equations are provided. In this algorithm, the heterogeneity of nodes, the expense for maintaining replicas and the engaged space have been considered. Under these restricted conditions, this algorithm realizes high data reliability in a distributed system. The results of case analysis prove the feasibility of the algorithm.
Cited by 1
Shu Zhang · Shenyang Gongye Daxue xuebao · 2011
In order to optimize the survival time of node in network,a Pareto optimal centralized algorithm for solving the lifetime of wireless sensor network was proposed based on the features of wireless sensor network.Entropy is the measurement of system average degree.By proving the equivalent relationship between maximum entropy function and Pareto optimal solution of sensor network lifetime,a dynamic programming model for solving the maximum entropy function of sensor network was established.The complex multi-objective linear programming was transformed into single-objective dynamic programming.The theoretical analysis and simulation results show that new algorithm can obtain the Pareto optimal solution quickly and efficiently and achieve the purpose of optimizing the sensor network lifetime.It improves the feasibility of system and reduces the computational complexity.
Cited by 0
Jun Song, Yugang Niu, Yuanyuan Zou · 2015
This paper investigates the uniformly ultimate boundedness (UUB) of an identifier-based adaptive dynamic programming (ADP) algorithm proposed in [7]. It is demonstrated that the estimation errors of weights in both critic and action networks are UUB during iteration learning. Moreover, a selection method on learning rates is also given.
Cited by 2
Aihua Yin, Chong Chen, Dongping Hu et al. · Communications in computer and information science · 2020
Cited by 1
Alexander Yumaganov, Anton Agafonov, Vladislav Myasnikov · Lecture notes in business information processing · 2021
Cited by 4
BERTOCCO, MATTEO, MIAN, GIAN ANTONIO, OFFELLI, CARLO · Research Padua Archive (University of Padua) · 1990
Cited by 0
Robert G. Haight, J. Douglas Brodie, Walter G. Dahms · Forest Science · 1985
Abstract Single-tree/distance-independent stand simulators can be adapted for dynamic programming (DP) optimization of thinnings and rotation age. Efficiency requires classification of trees for all but very short tree lists. Thinning types in conjunction with discrete basal-area, number-of-trees, and age classes define a DP network that allows the comparison of yields from more thinning regimes that have wider differences in sequences of diameter distributions than do networks without the thinning type dimension. Methods and results are demonstrated for a lodgepole pine ( Pinus contorta Doug. ex Loud.) simulator developed for Central and Eastern Oregon. Optimal management regimes developed with and without quality premiums produce stands with high vigor indices and high resistance to attack by mountain pine beetle ( Dendroctonus ponderosae Hopkins), suggesting that stocking control can be used to simultaneously maximize both net discounted revenue and insect resistance. However, managing stands with a maximum diameter limit to prevent attack substantially reduces net revenue when market premiums exist for large diameter trees. Forest Sci. 31:321-330.
Cited by 43Open access
Zhang Jian-xiang, Xusheng Gan, Jingjuan Sun et al. · SHILAP Revista de lepidopterología · 2020
The dynamic planning of the training airspace is of great significance for improving the utilization rate of the airspace, improving the efficiency of military training, and alleviating the contradiction between military and civilian air. The spatial dynamic programming problem is processed in stages, and the total occupation time is minimized by the optimal scheme of each stage. Aiming at the dynamic programming problem in each stage, on the basis of analyzing the complexity of the problem, the spatial planning model is constructed, and the genetic-discrete particle swarm optimization(DPSO) algorithm is proposed. By integrating the crossover and mutation ideas in the genetic algorithm, the DPSO algorithm’s ability to get rid of the local optimal solution is improved, and the convergence speed and accuracy of the algorithm are improved. At the same time, in order to ensure the diversity of population, the adaptive crossover operator and mutation operator are designed to ensure the individual feasibility, and the Gantt chart is used to represent the whole spatial planning process. Finally, the improved genetic-particle swarm optimization algorithm is used as an example. Compared with the traditional particle swarm optimization, the results show that the algorithm is of better results and faster convergence speed.
Cited by 0Open access
Moura, Leonardo Fernando dos Santos · Lume (Universidade Federal do Rio Grande do Sul) · 2013
This report describes an algorithm for the Unbounded Knapsack Problem based on the algorithm EDUK (Efficient Dynamic Programming for the Unbounded Knapsack Problem). EDUK takes advantage of the problem properties of dominance and periodicity to speed up computation. This algorithm is compared with other implementations and it is tested both with randomly generated instances and with instances generated from a delayed column generation for the Cutting Stock Problem. This report also contains an analysis of unbounded knapsack instances.
Cited by 0Open access
Meritxell Vinyals, Thomas Voice, Sarvapali D. Ramchurn et al. · arXiv (Cornell University) · 2013
We present a new Dynamic Programming (DP) formulation of the Coalition Structure Generation (CSG) problem based on imposing a hierarchical organizational structure over the agents. We show the efficiency of this formulation by deriving DyPE, a new optimal DP algorithm which significantly outperforms current DP approaches in speed and memory usage. In the classic case, in which all coalitions are feasible, DyPE has half the memory requirements of other DP approaches. On graph-restricted CSG, in which feasibility is restricted by a (synergy) graph, DyPE has either the same or lower computational complexity depending on the underlying graph structure of the problem. Our empirical evaluation shows that DyPE outperforms the state-of-the-art DP approaches by several orders of magnitude in a large range of graph structures (e.g. for certain scalefree graphs DyPE reduces the memory requirements by $10^6$ and solves problems that previously needed hours in minutes).
Cited by 4Open access
Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 1994
Cited by 0
Biing-Feng Wang · Theoretical Computer Science · 2023
Cited by 0