Eshagh Kargar, Ville Kyrki · arXiv (Cornell University) · 2024
Offline reinforcement learning allows training reinforcement learning models on data from live deployments. However, it is limited to choosing the best combination of behaviors present in the training data. In contrast, simulation environments attempting to replicate the live environment can be used instead of the live data, yet this approach is limited by the simulation-to-reality gap, resulting in a bias. In an attempt to get the best of both worlds, we propose a method that combines an imperfect simulation environment with data from the target environment, to train an offline reinforcement learning policy. Our experiments demonstrate that the proposed method outperforms state-of-the-art approaches CQL, MOPO, and COMBO, especially in scenarios with diverse and challenging dynamics, and demonstrates robust behavior across a variety of experimental conditions. The results highlight that using simulator-generated data can effectively enhance offline policy learning despite the sim-to-real gap, when direct interaction with the real-world is not possible.
Cited by 0Open access
Andrei Sleptchenko, Hasan Hüseyin Turan · Encyclopedia of Optimization · 2022
Cited by 0
Leng-feng Lee, Student Member, Venkat N. Krovi · 2006
Abstract — Rehabilitation is a complex multifaceted process with complexity and variability that depends not only on human patients and/or specialized equipment but also on the nature of their functional interaction. Rapid and effective customization of the functional interactions between the patient and the rehabilitation device thus becomes critical for any rehabilitation program. Two principal dimensions govern the effectiveness of such functional interactions: geometric placement of user-device (ergonomics) and exercise selection and performance (regimen). In this paper, we discuss aspects of creation of a Virtual Design Environment, leveraging tools from musculoskeletal analysis, optimization, and simulation-based design, which will permit a therapist to rapidly evaluate and systematically customize various candidate rehabilitation programs. Specifically, this framework: (i) permits study of parametric performance variability due to ergonomic or regimen variability; and (ii) facilitates use of all design tools such as optimization to determine the best program. We illustrate various aspects of this customization using an illustrative case-study of a motor-rehabilitation haptic virtual driving environment. E I.
Cited by 0
Tahmasebi, Farhang, Zach, Robert, Schuß, Matthias et al. · UCL Discovery (University College London) · 2012
Research on performance simulation deployment opportunities in the building operation phase has recently gained on momentum. Specifically, simulation routines have been successfully applied in the conception and implementation of predictive methods for building systems control. Needless to say, the quality of such a predictive control system depends on the reliability of the integrated simulation models. Thus, to ensure that predictions are dependable, the incorporated simulation models must be calibrated. Moreover, given the dynamic nature of building operation and the boundary conditions, some input parameters of the model may have to be subjected to calibration more frequently. Hence, the calibration task cannot be approached as a one-time activity. Rather, it needs to be conducted on a regular and systematic basis. Given this background, this paper examines the potential of recurrent optimization-based simulation model calibration of an office area. The results displayed a noticeable but not fully consistent improvement of the predictive potency of the calibrated model.
Cited by 6
Shawn Gano · Figshare · 2022
Simulation-based design optimization is playing an increasingly prominent role in the design of everything from spacecraft to consumer products. Applying nonlinear optimization techniques to simulation-based design becomes prohibitively expensive as computer models become more complex and increase in fidelity. A common engineering practice is to drive the preliminary design process using lower fidelity models as surrogates of expensive high fidelity simulations. Higher fidelity models are then used in the final design stages to refine the design. However, using automated optimization methods at this stage may still require enormous computational resources. Recently, variable fidelity schemes have been developed to address this problem by incorporating both models into one optimization framework. In these methods the low fidelity models are scaled to approximate the high fidelity simulations. This scaling allows the optimization to be performed using mainly low fidelity function calls, reducing the overall computational cost, while requiring only a few high fidelity evaluations to update the scaling function. Currently, two main scaling varieties are used: first order multiplicative and first order additive. In the multiplicative approach the low fidelity model is multiplied by a scaling function to approximate the high fidelity model; similarly, in the additive approach a scaling function is added to the low fidelity model. <p>The focus of this dissertation is on improving the efficiency and applicability of variable fidelity optimization algorithms. Highlights of original contributions made in this research include: (1) An adaptive hybrid scaling method that relieves designers from having to choose emph{a priori} which scaling method, multiplicative or additive, is most suitable to their problem with limited information. (2) Second order scaling methods which use approximate Hessian information, resulting in super-linear convergence rates. (3) A kriging-based global scaling method, which uses past design information to improve the global accuracy of the scaling model and was shown to reduce the computational cost of optimization by over 60% compared to single fidelity methods. (4) A metamodel update management strategy to reduce the cost of using kriging metamodels sequentially in large design problems. (5) Extension of the variable fidelity framework to solve reliability based design problems, which significantly lowers the computational cost, compared to traditional methods.</p>
Cited by 5Open access
Oleh Kusyy · Kobra (Universitätsbibliothek Kassel) · 2018
Cited by 0Open access
Dimitris Bertsimas, Omid Nohadani, Kwong Meng Teo · Operations Research · 2009
In engineering design, an optimized solution often turns out to be suboptimal when errors are encountered. Although the theory of robust convex optimization has taken significant strides over the past decade, all approaches fail if the underlying cost function is not explicitly given; it is even worse if the cost function is nonconvex. In this work, we present a robust optimization method that is suited for unconstrained problems with a nonconvex cost function as well as for problems based on simulations, such as large partial differential equations (PDE) solver, response surface, and Kriging metamodels. Moreover, this technique can be employed for most real-world problems because it operates directly on the response surface and does not assume any specific structure of the problem. We present this algorithm along with the application to an actual engineering problem in electromagnetic multiple scattering of aperiodically arranged dielectrics, relevant to nanophotonic design. The corresponding objective function is highly nonconvex and resides in a 100-dimensional design space. Starting from an “optimized” design, we report a robust solution with a significantly lower worst-case cost, while maintaining optimality. We further generalize this algorithm to address a nonconvex optimization problem under both implementation errors and parameter uncertainties.
Cited by 139
Jing Li, Zissimos P. Mourelatos, Michael Kokkolaras et al. · KTH Publication Database DiVA (KTH Royal Institute of Technology) · 2010
Computational simulation models support a rapid design process. Given model approximation and operating conditions uncertainty, designers must have confidence that the designs obtained using simulations will perform as expected. This paper presents a methodology for validating designs as they are generated during a simulation-based optimization process. Current practice focuses on validation of simulation models throughout the entire design space. In contrast, the proposed methodology requires validation only at design points generated during optimization. The goal of such validation is confidence in the resulting design rather than in the underlying simulation model. The proposed methodology is illustrated on a simple cantilever beam design subject to vibration.
Cited by 26Open access
Yan Wei, Hongxiang Wang · 2007
Due to rapid development of container terminal management, the berth allocation problem (BAP) becomes critically important to the operational efficiency of a container terminal. However, intensive research wok has currently approached this problem, involving the berthing stage only. To resolve this impact, a novel BAP strategy is investigated in this study. In this respect, a mathematical model is first established based on the quay length of a container terminal. Afterwards, heuristic algorithms are developed for simulating and optimizing the dynamic berth allocation strategy with regard to yard planning. Finally, a case on a specific container terminal has been used to illustrate the feasibility and effectiveness of the proposed approach. Further elaborated, the simulation results have also been presented based on a comparison between the simulation models before and after optimization.
Cited by 0
Grasso, Giulia · KTH Publication Database DiVA (KTH Royal Institute of Technology) · 2023
Nowadays, it has become crucial to transform company production processes in order to reduce carbon emissions. Therefore, the company LKAB is working to make the production process of iron ore pellets fossil-free. In particular, this thesis project focuses on the pellet induration stage and addresses the mathematical optimization of the process. In particular, the idea is to combine state-of-the-art optimization algorithms with simulation software. This thesis aims to address the problem of multi-objective optimization within the context of simulation-based scenarios, i.e., the aim is to study Simulation-Based Multi-Objective Optimization Problems. The primary focus of this thesis is to thoroughly investigate and compare three different algorithms applied to two distinct problem formulations. By doing so, we aim to gain valuable insights into the suitability of different approaches and evaluate the algorithms' performance in achieving the desired objectives.
Cited by 0Open access
Philipp Zmijewski, Nicolas Meseth · 2023
SimBO is a flexible framework for optimizing discrete event-driven simulations (DES) using sequential optimization algorithms. While specifically designed for Bayesian Optimization (BO) in the context of DES, SimBO can be applied to any black-box problem with other optimization algorithms. The framework consists of four encapsulated components - the black-box problem, the sequential optimization algorithm, a database for experiment configuration and results, and a web-based graphical user interface - that communicate via well-defined interfaces. Each component can be run in different environments, allowing for cooperation between different hardware- and software configurations. In our research context, SimBO’s architecture enabled BO algorithms to be run on a high-performance cluster with GPU support, while the simulation is executed on a local Windows machine using the Simio simulation software. The framework’s flexibility also makes it suitable for evolving from a research-focused tool to a production-ready, cloud-based optimization tool for modern algorithms.
Cited by 0Open access
Raid Al‐Aomar · 2003
This paper describes a methodology for solving parameter design (PD) problems in production and business systems of considerable complexity. The solution is aimed at determining optimum settings to system critical parameters so that each system response is at its optimum performance level with least amount of variability. When approaching such problem, analysts are often faced with four major challenges: representing the complex parameter design problem, utilizing an effective search method that is able to explore the problem's complex and large domain, making optimization decisions based on multiple and, often, conflicting objectives, and handling the stochastic variability of system response as an integral part of the search method. to tackle such challenges, this paper proposes a solution methodology that integrates four state-of-the-art modules of proven methods: simulation modeling (SM), genetic algorithm (GA), entropy method (EM), and robustness module (RM).
Cited by 26
Peter Marbach, John N. Tsitsiklis · 2002
We propose a simulation-based algorithm for optimizing the average reward in a Markov reward process that depends on a set of parameters. As a special case, the method applies to Markov decision processes where optimization takes place within a parametrized set of policies. The algorithm involves the simulation of a single sample path, and can be implemented online. A convergence result (with probability 1) is provided.
Cited by 28
Peter Marbach, John N. Tsitsiklis · IEEE Transactions on Automatic Control · 2001
This paper proposes a simulation-based algorithm for optimizing the average reward in a finite-state Markov reward process that depends on a set of parameters. As a special case, the method applies to Markov decision processes where optimization takes place within a parametrized set of policies. The algorithm relies on the regenerative structure of finite-state Markov processes, involves the simulation of a single sample path, and can be implemented online. A convergence result (with probability 1) is provided.
Cited by 317Open access
Alison Cozad, Nikolaos V. Sahinidis, David C. Miller · AIChE Journal · 2014
A central problem in modeling, namely that of learning an algebraic model from data obtained from simulations or experiments is addressed. A methodology that uses a small number of simulations or experiments to learn models that are as accurate and as simple as possible is proposed. The approach begins by building a low‐complexity surrogate model. The model is built using a best subset technique that leverages an integer programming formulation to allow for the efficient consideration of a large number of possible functional components in the model. The model is then improved systematically through the use of derivative‐free optimization solvers to adaptively sample new simulation or experimental points. Automated learning of algebraic models for optimization (ALAMO), the computational implementation of the proposed methodology, along with examples and extensive computational comparisons between ALAMO and a variety of machine learning techniques, including Latin hypercube sampling, simple least‐squares regression, and the lasso is described. © 2014 American Institute of Chemical Engineers AIChE J , 60: 2211–2227, 2014
Cited by 439
Brenner, M. · Fraunhofer-Publica (Fraunhofer-Gesellschaft) · 1998
S.79-96
Cited by 0
H. Yin, Xin Pan, Yuning Wang et al. · 2024
Cited by 0
L. F. Perrone, F. P. Wiel, J. Liu et al. · 2008
Traditionally, a simulation-based optimization (SO) system is designed as a black-box in which the internal details of the optimization process is hidden from the user and only the final optimization solutions are presented. As the complexity of the SO systems and the optimization problems to be solved increases, instrumentation – a technique for monitoring and controlling the SO processes – is becoming more important. This paper proposes a white-box approach by advocating the use of instrumentation components in SO systems, based on a component-based architecture. This paper argues that a number of advantages, including efficiency enhancement, gaining insight from the optimization trajectories and higher controllability of the SO processes, can be brought out by an on-line instrumentation approach. This argument is supported by the illustration of an instrumentation component developed for a SO system designed for solving real-world multi-objective operation scheduling problems.
Cited by 0
Mihailo R. Jovanović, Ermin Husak · Lecture notes in networks and systems · 2019
Cited by 6
J. Wray, A. Buss, Jérémy Clément Salmeron · 2017
Cited by 0