Hyperparameter Optimization
Matthias Feurer, Frank Hutter · The Springer series on challenges in machine learning · 2019
Cited by 1,147
Search 250 million research papers
Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.
About 10,000 papers for “Hyperparameter optimization”
Hyperparameter optimization.
Ann-Kristin Becker (9099912), Till Ittermann (528482), Markus Dörr (13132537) et al. · Figshare · 2022
<p>Prediction results from the grid-based hyperparameter optimization of the random forest model using 10-fold nested cross-validation. Rows show results for varying values of the parameter nodesize (tested values: 15, 40, 65), and columns show results for varying values of the maximal number of terminal nodes (maxnodes, tested values: 15, 40, 65). The parameter mtry was optimized separately and set to 5.</p> <p>(TIF)</p>
Cited by 0Open access
Hyperparameter Optimization
Toshihiko Yanase · The Journal of The Institute of Image Information and Television Engineers · 2023
Cited by 0Open access
Hyperparameter Optimization
Taşkın Kavzoğlu, Brandt Tso, Paul M. Mather · 2024
Cited by 0
Hyperparameter optimization.
Maria Kalweit (11047560), Ulrich A. Walker (11047563), Axel Finckh (3918683) et al. · Figshare · 2021
<p>Configuration spaces for all approaches. The best performing parameter setting is shown in bold.</p> <p>(DOCX)</p>
Cited by 0Open access
Hyperparameter optimization results.
Minan Tang (20389941), Hongjie Wang (54357), Jiandong Qiu (20389944) et al. · Figshare · 2024
<div><p>The large-scale integration of offshore wind power into the power grid has brought serious challenges to the power system power quality. Aiming at the problem of power quality disturbance detection and classification, this paper proposes a novel algorithm based on fast S-transform and crested porcupine optimizer (CPO) optimized CNN. Firstly, the intrinsic mechanism and waveform characteristics of offshore wind power grid-connected disturbances are analyzed, and the simulated disturbance signals are feature extracted and time-frequency diagrams are obtained by fast S-transform. Secondly, the CPO algorithm is used to optimize the convolutional neural network and determine the best hyperparameters so that the classifier achieves the optimal classification performance. Then, the CPO-CNN classification model is used for feature extraction and feature selection of the time-frequency diagrams and classification of multiple power quality disturbances. Finally, a simulation experimental platform is established based on MATLAB to perform simulation verification and comparative analysis of power quality disturbance classification. The experimental results show that the model established in this paper is effective, and the classification accuracy is improved by 3.47% compared with the CNN method, which can accurately identify the power quality disturbance signals, and then help to assess and control the power quality problems.</p></div>
Cited by 0Open access
Hyperparameter optimization results.
Jiayue Zhang (7832669), Xinxin Yi (3932963), Heng Wang (76752) · Figshare · 2025
<div> Inspection and diagnosis of construction projects involves health monitoring of buildings and related facilities, and the utilization of renewable energy sources, such as solar energy, is critical to the smooth operation of modern construction projects. The detection of solar cell defects is related to the reliability and efficiency of building photovoltaics and has become an area of interest. Existing deep learning-based solar cell defect detection models significantly improve the accuracy of solar cell defect detection, however, deep learning-based solar cell defect detection models ignore the effect of network hyperparameters on their model performance. In this study, the hybrid model CMNS-YOLO, which combines the crawfish optimization algorithm with the MNS-YOLO model, is proposed to achieve the ultimate detection accuracy. First, Mamba-Like Linear Attention is introduced to design the C2f-MLLA module to improve the target feature representation capability of solar cell sheet defects; second, Bidirectional feature pyramid frequency aware feature fusion network is designed to enhance the recovery ability of target detail features as well as the fusion ability of image features; then ShapeIoU is used to solve the target aspect ratio misalignment problem and construct the improved MNS-YOLO network; finally, COA is utilized to adjust the parameters of the MNS-YOLO network. Experimental results on the PV-Multi-Defect and PVELAD datasets show that compared with the baseline model, the detection accuracy of the proposed model on the two datasets is improved by 6.3% and 2.3% while maintaining the lightweight characteristics of the model. Therefore, the proposed method has considerable potential in the field of solar cell defect detection. </div>
Cited by 0Open access
Efficient hyperparameter optimization
Jiří Krejčí · Digital Repository (National Repository of Grey Literature) · 2024
Hyperparameter optimization significantly impacts model performance and substantial effort went into the development of robust and efficient algo- rithms for this task. Our research found that several new algorithms utilizing partial evaluations have been published recently. However, it is not clear from the literature how the algorithms perform in various scenarios. In this thesis, we compared the leading algorithms through experiments on diverse tasks, including tabular benchmarks and real-world deep-learning problems, with a special focus on healthcare datasets. The results show that the recent multi-fidelity techniques outperform random search. Nevertheless, no single algorithm consistently excelled across all problems, highlighting the need for ongoing comparison studies in hyperparameter optimization.
Cited by 0
Advancing hyperparameter optimization
Schneider, Lennart Paul · Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München) · 2025
Hyperparameter optimization (HPO) is a fundamental aspect of machine learning (ML), directly influencing model performance and adaptability. As a computationally expensive black-box optimization problem, HPO requires efficient algorithms to identify optimal hyperparameter configurations. This thesis advances the field of HPO along three key dimensions: foundational insights, HPO in the presence of more than one objective, and algorithmic innovations through benchmarking. First, we revisit resampling strategies for performance estimation, demonstrating both theoretically and empirically that reshuffling resampling splits across hyperparameter configurations enhances generalization. Additionally, we conduct an in-depth analysis of HPO validation landscapes, revealing characteristics such as low multimodality and broad plateaus that differentiate them from conventional black-box optimization benchmarks. Second, we introduce novel algorithms for HPO in multi-objective and quality diversity settings. We propose a new approach for simultaneously optimizing model performance and interpretability, quantifying interpretability through feature sparsity, sparsity of interaction effects, and sparsity of non-monotone features. Furthermore, we bridge the field of quality diversity optimization with HPO, which allows us to discover diverse yet well-performing neural architectures that satisfy varying hardware constraints within a single optimization run. Third, we use benchmarking to drive algorithmic innovation and insights in HPO. We present YAHPO Gym, a scalable benchmarking suite supporting single-objective, multi-fidelity, and multi-objective HPO via surrogate benchmarks. Using this framework, we define new quality diversity problems inspired by HPO and develop a novel multi-fidelity optimization algorithm guided by programming by optimization principles. Additionally, we ablate a state-of-the-art neural architecture search algorithm to assess the impact of individual components and introduce a systematic approach for constructing synthetic black-box functions that admit specific optimization landscape properties. By deepening our general understanding of HPO, proposing novel multi-objective and quality diversity optimization strategies, and developing scalable benchmarking tools, this thesis enhances the efficiency and effectiveness of HPO across diverse ML applications.
Cited by 0
Reproducible Hyperparameter Optimization
Hertel, Lars, Baldi, Pierre, Gillen, Daniel L. · Figshare · 2021
A key issue in current machine learning research is the lack of reproducibility. We illustrate what role hyperparameter search plays in this problem and how regular hyperparameter search methods can lead to a large variance in outcomes due to non-deterministic model training during hyperparameter optimization. The variation in outcomes poses a problem both for reproducibility of the hyperparameter search itself and comparisons of different methods each optimized using hyperparameter search. In addition, the fact that hyperparameter search may result in non-optimal hyperparameter settings may affect other studies since hyperparameter settings are often copied from previously published research. To remedy this issue we define the mean prediction error across model training runs as the objective for the hyperparameter search. We then propose a hypothesis testing procedure that makes inference on the mean performance of each hyperparameter setting and results in an equivalence class of hyperparameter settings that are not distinguishable in performance. We further embed this procedure into a group sequential testing framework to increase efficiency in terms of the average number of model training replicates required. Empirical results on machine learning benchmarks show that at equal computation the proposed method reduces the variation in hyperparameter search outcomes by up to 90 percent while resulting in equal or lower mean prediction errors when compared to standard random search and Bayesian optimization. Moreover, the sequential testing framework successfully reduces computation while preserving performance of the method. Code to reproduce the results is available in the supplementary materials and online<sup>1</sup>.
Cited by 0Open access
Hyperparameter optimization results.
Lihui Zhu (1783060), Yunfeng Yang (47638), Wenyue Yu (658311) et al. · Figshare · 2025
Bayesian optimization explored 30 configurations across learning rate, weight decay, dropout, model dimension (d_model), number of layers, and feedforward dimension, with nhead fixed at 8.
Cited by 0Open access
Tutorium "Hyperparameter Optimization"
Debus, Charlotte, Rüttgers, Alexander · elib (German Aerospace Center) · 2019
Part I: Theory \n- Train / development / test sets \n- Regularization techniques (dropout, L1/L2-regularization) \n- Optimization algorithms \n- Batch normalization \n- Grid search vs. random search vs. Bayesian optimization vs. gradient-based optimization vs. evolutionary optimization \n \nPart II: Hands-on "Hyperparameter Optimization for Improving Neural Networks " \n- Optimization of Jupyter notebooks using the Talos library for Keras \n- Good practice guidelines for hyperparameter tuning.
Cited by 0
Hyperparameter Optimization Results
Duncan Forster · Figshare · 2021
Hyperparameter optimization results across integration methods integrating three Schizosaccharomyces pombe networks. The chosen (best) hyperparameter combinations for each method are highlighted.<br>
Cited by 0Open access
Hyperparameter optimization results.
Yusuf A. Sha’aban (16052906) · Figshare · 2024
<div><p>There are global efforts to deploy Electric Vehicles (EVs) because of the role they promise to play in energy transition. These efforts underscore the e-mobility paradigm, representing an interplay between renewable energy resources, smart technologies, and networked transportation. However, there are concerns that these initiatives could burden the electricity grid due to increased demand. Hence, the need for accurate short-term load forecasting is pivotal for the efficient planning, operation, and control of the grid and associated power systems. This study presents robust models for forecasting half-hourly and hourly loads in the UK’s power system. The work leverages machine learning techniques such as Support Vector Regression (SVR), Artificial Neural Networks (ANN), and Gaussian Process Regression (GPR) to develop robust prediction models using the net imports dataset from 2010 to 2020. The models were evaluated based on metrics like Root Mean Square Error (RMSE), Mean Absolute Prediction Error (MAPE), Mean Absolute Deviation (MAD), and the Correlation of Determination (R<sup>2</sup>). For half-hourly forecasts, SVR performed best with an R-value of 99.85%, followed closely by GPR and ANN. But, for hourly forecasts, ANN led with an R-value of 99.71%. The findings affirm the reliability and precision of machine learning methods in short-term load forecasting, particularly highlighting the superior accuracy of the SVR model for half-hourly forecasts and the ANN model for hourly forecasts.</p></div>
Cited by 0Open access
Hyperparameter optimization results.
Takuma Shibahara (5437208), Chisa Wada (10825570), Yasuho Yamashita (10825573) et al. · Figshare · 2023
<p>This file stores the hyperparameter search ranges and their optimized values of each prediction model.</p> <p>(XLSX)</p>
Cited by 0Open access
Hyperparameter optimization results.
Yusuf A. Sha’aban (16052906) · Figshare · 2024
<div><p>There are global efforts to deploy Electric Vehicles (EVs) because of the role they promise to play in energy transition. These efforts underscore the e-mobility paradigm, representing an interplay between renewable energy resources, smart technologies, and networked transportation. However, there are concerns that these initiatives could burden the electricity grid due to increased demand. Hence, the need for accurate short-term load forecasting is pivotal for the efficient planning, operation, and control of the grid and associated power systems. This study presents robust models for forecasting half-hourly and hourly loads in the UK’s power system. The work leverages machine learning techniques such as Support Vector Regression (SVR), Artificial Neural Networks (ANN), and Gaussian Process Regression (GPR) to develop robust prediction models using the net imports dataset from 2010 to 2020. The models were evaluated based on metrics like Root Mean Square Error (RMSE), Mean Absolute Prediction Error (MAPE), Mean Absolute Deviation (MAD), and the Correlation of Determination (R<sup>2</sup>). For half-hourly forecasts, SVR performed best with an R-value of 99.85%, followed closely by GPR and ANN. But, for hourly forecasts, ANN led with an R-value of 99.71%. The findings affirm the reliability and precision of machine learning methods in short-term load forecasting, particularly highlighting the superior accuracy of the SVR model for half-hourly forecasts and the ANN model for hourly forecasts.</p></div>
Cited by 0Open access
Hyperparameter Optimization Results
Duncan Forster, Charlie Boone, Gary D. Bader et al. · Figshare · 2021
Hyperparameter optimization results across integration methods integrating three Schizosaccharomyces pombe networks. The chosen (best) hyperparameter combinations for each method are highlighted.<br>
Cited by 0Open access
PyHopper -- Hyperparameter optimization
Mathias Lechner, Ramin Hasani, Philipp Neubauer et al. · arXiv (Cornell University) · 2022
Hyperparameter tuning is a fundamental aspect of machine learning research. Setting up the infrastructure for systematic optimization of hyperparameters can take a significant amount of time. Here, we present PyHopper, a black-box optimization platform designed to streamline the hyperparameter tuning workflow of machine learning researchers. PyHopper's goal is to integrate with existing code with minimal effort and run the optimization process with minimal necessary manual oversight. With simplicity as the primary theme, PyHopper is powered by a single robust Markov-chain Monte-Carlo optimization algorithm that scales to millions of dimensions. Compared to existing tuning packages, focusing on a single algorithm frees the user from having to decide between several algorithms and makes PyHopper easily customizable. PyHopper is publicly available under the Apache-2.0 license at https://github.com/PyHopper/PyHopper.
Cited by 1Open access
Reproducible Hyperparameter Optimization
Lars Hertel, Pierre Baldi, Daniel L. Gillen · Figshare · 2021
A key issue in current machine learning research is the lack of reproducibility. We illustrate what role hyperparameter search plays in this problem and how regular hyperparameter search methods can lead to a large variance in outcomes due to non-deterministic model training during hyperparameter optimization. The variation in outcomes poses a problem both for reproducibility of the hyperparameter search itself and comparisons of different methods each optimized using hyperparameter search. In addition, the fact that hyperparameter search may result in non-optimal hyperparameter settings may affect other studies since hyperparameter settings are often copied from previously published research. To remedy this issue we define the mean prediction error across model training runs as the objective for the hyperparameter search. We then propose a hypothesis testing procedure that makes inference on the mean performance of each hyperparameter setting and results in an equivalence class of hyperparameter settings that are not distinguishable in performance. We further embed this procedure into a group sequential testing framework to increase efficiency in terms of the average number of model training replicates required. Empirical results on machine learning benchmarks show that at equal computation the proposed method reduces the variation in hyperparameter search outcomes by up to 90 percent while resulting in equal or lower mean prediction errors when compared to standard random search and Bayesian optimization. Moreover, the sequential testing framework successfully reduces computation while preserving performance of the method. Code to reproduce the results is available in the supplementary materials and online<sup>1</sup>.
Cited by 0Open access
Hyperparameter optimization results.
Moein E. Samadi (13804201), Sandra Kiefer (13804204), Sebastian Johaness Fritsch (13804207) et al. · Figshare · 2022
<p>The optimized hyperparameters resulted from grid-search cross-validation and Keras tuner for the supervised ML methods.</p> <p>(PDF)</p>
Cited by 0Open access