Vitercik, Ellen · Figshare · 2021
Algorithms from diverse domains often have tunable parameters that have a significant impact on performance metrics such as solution quality, runtime, and memory usage. Typically, the optimal setting depends intimately on the application domain at hand. Hand-tuning parameters is often tedious, time-consuming, and error-prone, so a burgeoning line of research has studied automated algorithm configuration via machine learning, where a training set of typical problem instances from the application at hand is used to find high-performing parameter settings.In this thesis, we help develop the theory and practice of automated algorithm configuration. We investigate both statistical and algorithmic questions. For example,how large should the training set be to ensure that an algorithm’s average performance over the training set is indicative of its future performance on unseen instances? How can we algorithmically find provably high-performing configurations? As we answer these questions, we analyze parameterized algorithms from a variety of domains, including:1. Integer programming. We study branch-and-bound algorithms for integer linear programming, the most widely-used tools for solving combinatorial and nonconvexproblems. Beyond answering the algorithmic and statistical questions above, we provide experiments demonstrating that no one parameter setting is optimal across all application domains, and that tuning parameters using ourapproach can have a significant impact on algorithmic performance. We also analyze integer quadratic programming approximation algorithms, which canbe used to find nearly-optimal solutions to a variety of combinatorial problems.2. Mechanism design. A mechanism is a special type of algorithm that plays a crucial role in economics and political science. A mechanism’s purpose is to help a set of agents come to a collective decision. In economics, for example, a mechanism might dictate how a set of items should be split among the agents, given their values for those items. Mechanisms often have tunable parameters that impact, for example, their revenue. No one setting is optimal across all mechanism design scenarios. In this thesis, we analyze sales mechanisms, where the goal is to maximize revenue, and voting mechanisms, where the goal is to maximize the agents’ total value for the outcome the mechanism selects.3. Computational biology. Algorithms from computational biology are often highly parameterized, and understanding which parameter settings are optimal in which scenarios is an active area of research. We analyze parameterized algorithms for several fundamental problems from computational biology, including sequence alignment, RNA folding, and predicting topologically associating domains in DNA sequences. The key challenge from both an algorithmic and statistical perspective is that across these three diverse domains, an algorithm’s performance is an erratic function of its parameters. This is because a small tweak to the parameters can cause a cascade of changes in the algorithm’s performance. We develop tools for analyzing andoptimizing these volatile performance functions, which we use to provide parameter tuning procedures and strong statistical guarantees.
Cited by 0Open access
Thomas Stützle, Manuel López‐Ibáñez, Leslie Pérez Cáceres · Proceedings of the Genetic and Evolutionary Computation Conference Companion · 2022
tutorial Share on Automated algorithm configuration and design Authors: Thomas Stützle ULB, Brussels, Belgium ULB, Brussels, BelgiumView Profile , Manuel López-Ibáñez University of Malaga University of MalagaView Profile , Leslie Pérez-Cáceres Pontifical Catholic University of Valparaíso Pontifical Catholic University of ValparaísoView Profile Authors Info & Claims GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionJuly 2022 Pages 997–1019https://doi.org/10.1145/3520304.3533663Published:19 July 2022Publication History 0citation43DownloadsMetricsTotal Citations0Total Downloads43Last 12 Months43Last 6 weeks9 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Cited by 7
Dimitri Weiß, Elias Schede, Kevin Tierney · Journal of Heuristics · 2025
Abstract Solvers contain parameters that influence their performance and these must be set by the user to ensure that high-quality solutions are generated, or optimal solutions are found quickly. Manually setting these parameters is tedious and error-prone, since search spaces may be large or even infinite. Existing approaches to automate the task of algorithm configuration (AC) make use of a single machine learning model that is trained on previous runtime data and used to create or evaluate promising new configurations. We combine a variety of successful models from different AC approaches into an ensemble that proposes new configurations. To this end, each model in the ensemble suggests configurations and a hyper-configurable selection algorithm chooses a subset of configurations to match the amount of computational resources available. We call this approach Selector , and we examine its performance against the state-of-the-art AC methods PyDGGA and SMAC, respectively. The new configurator will be made available as an open source software package.
Cited by 1Open access
Holger H. Hoos · 2011
Cited by 195
Yuan, Zhi · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · 2019
Algorithms for solving hard optimization problems usually have a number of parameters that greatly influence the algorithm performance. Instead of manual or ad hoc methods to adjust the parameters, using automatic tools for configuring such parameters have been proved to be crucial for deriving high-performing algorithms. Automatic algorithm configuration can be regarded as a black-box mixed-discrete-continuous variable optimization problem. Solving such problem usually requires two components for the two subtasks: a black-box search method that generates candidate configurations, and an evaluation method that evaluate the quality of the generated configuration under stochasticity.In this work, we define two frameworks, namely, iterated selection and post-selection, for combining search method and evaluation method for algorithm configuration. Through extensive literature review, all the established algorithm configurators can be identified to be under the iterated selection framework. We have extended an established evaluation method, the statistical racing method, with an ad-hoc iterated search method for configuring categorical and conditional parameters, as well as combined the racing method with established black-box search methods, such as MADS and CMA-ES, following the iterated selection framework. We have introduced state-of-the-art black-box optimizers such as CMA-ES, BOBYQA, and Nelder-Mead simplex to solving the problem of automatic algorithm configuration, and demonstrated how state-of-the-art configurator can be obtained by hybridizing these black-box search method with statistical racing method following the post-selection framework. The best settings for devising post-selection configurators are empirically analyzed in depth. We have identified and proposed a decreasing population CMA-ES with post-selection as the state-of-the-art algorithm configurator across extensive benchmark configuration problems with various dimensions and configuration budget.
Cited by 0Open access
Elias Schede, Jasmin Brandt, Alexander Tornede et al. · Journal of Artificial Intelligence Research · 2022
Algorithm configuration (AC) is concerned with the automated search of the most suitable parameter configuration of a parametrized algorithm. There is currently a wide variety of AC problem variants and methods proposed in the literature. Existing reviews do not take into account all derivatives of the AC problem, nor do they offer a complete classification scheme. To this end, we introduce taxonomies to describe the AC problem and features of configuration methods, respectively. We review existing AC literature within the lens of our taxonomies, outline relevant design choices of configuration approaches, contrast methods and problem variants against each other, and describe the state of AC in industry. Finally, our review provides researchers and practitioners with a look at future research directions in the field of AC.
Cited by 57Open access
Elias Schede, Jasmin Brandt, Alexander Tornede et al. · arXiv (Cornell University) · 2022
Algorithm configuration (AC) is concerned with the automated search of the most suitable parameter configuration of a parametrized algorithm. There is currently a wide variety of AC problem variants and methods proposed in the literature. Existing reviews do not take into account all derivatives of the AC problem, nor do they offer a complete classification scheme. To this end, we introduce taxonomies to describe the AC problem and features of configuration methods, respectively. We review existing AC literature within the lens of our taxonomies, outline relevant design choices of configuration approaches, contrast methods and problem variants against each other, and describe the state of AC in industry. Finally, our review provides researchers and practitioners with a look at future research directions in the field of AC.
Cited by 6Open access
Quentin Renau, Johann Dréo, Asher Peres et al. · Zenodo (CERN European Organization for Nuclear Research) · 2022
This upload is a dataset companion to a paper in the Real World Application track of the GECCO 2022 conference. The dataset includes a binary container for the objective function used in the paper as well as the corresponding experimental logs.
Cited by 1Open access
Holger H. Hoos, Frank Neumann, Heike Trautmann · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2017
This report documents the programme and the outcomes of Dagstuhl Seminar 16412 "Automated Algorithm Selection and Configuration", which was held October 9--14, 2016 and attended by 34 experts from 10 countries. Research on automated algorithm selection and configuration has lead to some of the most impressive successes within the broader area of empirical algorithmics, and has proven to be highly relevant to industrial applications. Specifically, high-performance algorithms for cnp-hard problems, such as propositional satisfiability (SAT) and mixed integer programming (MIP), are known to have a huge impact on sectors such as manufacturing, logistics, healthcare, finance, agriculture and energy systems, and algorithm selection and configuration techniques have been demonstrated to achieve substantial improvements in the performance of solvers for these problems. Apart from creating synergy through close interaction between the world's leading groups in the area, the seminar pursued two major goals: to promote and develop deeper understanding of the behaviour of algorithm selection and configuration techniques and to lay the groundwork for further improving their efficacy. Towards these ends, the organisation team brought together a group of carefully chosen researchers with strong expertise in computer science, statistics, mathematics, economics and engineering; a particular emphasis was placed on bringing together theorists, empiricists and experts from various application areas, with the goal of closing the gap between theory and practice.
Cited by 7Open access
Slabbekoorn, J.R. · TU/e Research Portal
Cited by 0
Slabbekoorn, JR René · TU/e Research Portal (Eindhoven University of Technology) · 2016
Cited by 0
König, H.M.T., Hoos, H.H., Rijn, J.N. van · Leiden Repository (Leiden University) · 2021
Cited by 0
Jeroen Rook · 2025
Cited by 0
Оливер Де Лоубенк · Open MIND · 2025
В ходе выполнения работы, был разработан алгоритм выбора конфигурации фланцевых соединений, включающий создание структурированной базы данных стандартных фланцев, формализацию математических моделей расчета, генерацию параметрической 3D-модели и интеграцию расчетного модуля в ПО SolidWorks с помощью макросов, реализованных на языке VBA. В рамках разработки были учтены конструктивные и эксплуатационные параметры фланцев, физико-механические характеристики материалов, а также эксплуатационные нагрузки, включая рабочее давление, температурные воздействия и тип рабочей среды. Алгоритм реализован в виде программного инструмента, основанного на модульной архитектуре, с возможностью автоматического ввода исходных данных пользователем, выполнения расчетов по встроенным формулам и построения параметризованной 3D-модели фланца. Использование интерфейса SolidWorks API позволило связать расчетную часть с геометрическим моделированием, обеспечив генерацию адаптивной модели, полностью соответствующей полученным параметрам. Встроенные средства SolidWorks Simulation были задействованы для выполнения предварительного анализа напряженно-деформированного состояния, что позволило провести верификацию проектных решений на этапе проектирования.
Cited by 0
Yuan, Zhi · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · 2019
Algorithms for solving hard optimization problems usually have a number of parameters that greatly influence the algorithm performance. Instead of manual or ad hoc methods to adjust the parameters, using automatic tools for configuring such parameters have been proved to be crucial for deriving high-performing algorithms. Automatic algorithm configuration can be regarded as a black-box mixed-discrete-continuous variable optimization problem. Solving such problem usually requires two components for the two subtasks: a black-box search method that generates candidate configurations, and an evaluation method that evaluate the quality of the generated configuration under stochasticity.In this work, we define two frameworks, namely, iterated selection and post-selection, for combining search method and evaluation method for algorithm configuration. Through extensive literature review, all the established algorithm configurators can be identified to be under the iterated selection framework. We have extended an established evaluation method, the statistical racing method, with an ad-hoc iterated search method for configuring categorical and conditional parameters, as well as combined the racing method with established black-box search methods, such as MADS and CMA-ES, following the iterated selection framework. We have introduced state-of-the-art black-box optimizers such as CMA-ES, BOBYQA, and Nelder-Mead simplex to solving the problem of automatic algorithm configuration, and demonstrated how state-of-the-art configurator can be obtained by hybridizing these black-box search method with statistical racing method following the post-selection framework. The best settings for devising post-selection configurators are empirically analyzed in depth. We have identified and proposed a decreasing population CMA-ES with post-selection as the state-of-the-art algorithm configurator across extensive benchmark configuration problems with various dimensions and configuration budget.
Cited by 0Open access
Jeroen; id_orcid 0000-0002-3921-0107 Rook · University of Twente Research Information · 2025
This thesis presents significant progress in automating algorithm configuration and selection for scenarios involving multiple performance objectives – an important step toward building more trustworthy, adaptable, transparent, and resource-efficient artificial intelligence systems. By advancing meta-algorithmic methods, it makes a substantial contribution to the field of artificial intelligence.<br/> The central focus is on managing conflicting objectives, such as computational efficiency, accuracy, and explainability, which frequently arise in real-world applications. Multi-objective scenarios require finding trade-off between various goals simultaneously – improving on one often comes at the cost of the other goals. While existing automated frameworks excel in optimising for single objectives, they struggle in these<br/>multi-objective contexts. This work proposes extensions to current paradigms by incorporating multi-objective optimisation techniques into automated algorithm design.<br/> A further challenge addressed is the adaptability of these methods to dynamic environments, where problem characteristics may shift over time and static models become inadequate. Ensuring algorithmic robustness under changing conditions is essential for real-world deployment.<br/> Beyond the development of meta-algorithmic frameworks, the thesis introduces new approaches for evaluating performance in continuous multi-objective optimisation. It presents a novel evaluation metric that balances convergence in the objective space with diversity in the decision space, which are two aspects often overlooked by traditional measures. In addition, a robust statistical ranking method is proposed,<br/>offering more reliable algorithm comparisons in the face of instance variability and stochastic noise.<br/> The effectiveness of the proposed methods is demonstrated through extensive experiments across various problems from artificial intelligence. Case studies include multi-modal multi-objective optimisation and sparse neural network training, highlighting practical trade-offs between competing goals, such as accuracy and efficiency.<br/> Finally, to support broader impact and adoption, the developed approaches are made accessible through open-source software tools, enabling their integration in both academic research and applied industry settings.
Cited by 0Open access
Matthias Carnein, Heike Trautmann, Albert Bifet et al. · Lecture notes in computer science · 2020
Cited by 13Open access
Elias Schede, Jasmin Brandt, Alexander Tornede et al. · 2023
Algorithm configuration (AC) is concerned with the automated search of the most suitable parameter configuration of a parametrized algorithm. There are currently a wide variety of AC problem variants and methods proposed in the literature. Existing reviews do not take into account all derivatives of the AC problem, nor do they offer a complete classification scheme. To this end, we introduce taxonomies to describe the AC problem and features of configuration methods, respectively. Existing AC literature is classified and characterized by the provided taxonomies.
Cited by 1Open access
Rens van Haveren, B. Heijmen, Sebastiaan Breedveld · Radiotherapy and Oncology · 2019
Cited by 0Open access
Lin Xu, Frank Hutter, Holger H. Hoos et al. · 2012
Abstract. State-of-the-art mixed integer programming (MIP) solvers are highly parameterized. For heterogeneous and a priori unknown instance distributions, no single parameter configuration generally achieves consistently strong performance, and hence it is useful to select from a portfolio of different configurations. HYDRA is a recent method for using automated algorithm configuration to derive multiple configurations of a single parameterized algorithm for use with portfolio-based selection. This paper shows that, leveraging two key innovations, HYDRA can achieve strong performance for MIP. First, we describe a new algorithm selection approach based on classification with a non-uniform loss function, which significantly improves the performance of algorithm selection for MIP (and SAT). Second, by modifying HYDRA’s method for selecting candidate configurations, we obtain better performance as a function of training time. 1
Cited by 0