Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “hyperparameter optimization”

Hyperparameter optimization results obtained with Optuna

Rincón Cardeño, Oscar, Montoya-Noguera, Silvana, Guarin-Zapata, Nicolas et al. · Figshare · 2025

The top panel shows the optimization history, i.e., the evolution of the objective value across trials.<br>The bottom panels display the slice plots, illustrating the relationship between each hyperparameter (the activation function σ, the number of hidden layers 𝐿, the number of neurons per layer 𝑁, and the learning rate α of the Adam optimizer) and the achieved objective value.

Cited by 0Open access

Raw outputs of the hyperparameter optimization

Lüken, Malte · Figshare · 2025

Raw outputs of the hyperparameter optimization from the paper:<br>"Generalization in neural posterior estimation: Case studies with the racing diffusion model".<br>Files:train_npe_trials.db: SQLite database with information over different Optuna trials.trials.csv: Optuna trials information in CSV format.

Cited by 0Open access

Meta-Surrogate Benchmarking for Hyperparameter Optimization

Aaron Klein, Zhenwen Dai, Frank Hutter et al. · arXiv (Cornell University) · 2019

Despite the recent progress in hyperparameter optimization (HPO), available benchmarks that resemble real-world scenarios consist of a few and very large problem instances that are expensive to solve. This blocks researchers and practitioners not only from systematically running large-scale comparisons that are needed to draw statistically significant results but also from reproducing experiments that were conducted before. This work proposes a method to alleviate these issues by means of a meta-surrogate model for HPO tasks trained on off-line generated data. The model combines a probabilistic encoder with a multi-task model such that it can generate inexpensive and realistic tasks of the class of problems of interest. We demonstrate that benchmarking HPO methods on samples of the generative model allows us to draw more coherent and statistically significant conclusions that can be reached orders of magnitude faster than using the original tasks. We provide evidence of our findings for various HPO methods on a wide class of problems.

Cited by 6Open access

LiDAR-in-the-Loop Hyperparameter Optimization

Félix Goudreault, Dominik Scheuble, Mario Bijelic et al. · 2023

LiDAR has become a cornerstone sensing modality for 3D vision. LiDAR systems emit pulses of light into the scene, take measurements of the returned signal, and rely on hardware digital signal processing (DSP) pipelines to construct 3D point clouds from these measurements. The resulting point clouds output by these DSPs are input to downstream 3D vision models - both, in the form of training datasets or as input at inference time. Existing LiDAR DSPs are composed of cascades of parameterized operations; modifying configuration parameters results in significant changes in the point clouds and consequently the output of downstream methods. Existing methods treat LiDAR systems as fixed black boxes and construct downstream task networks more robust with respect to measurement fluctuations. Departing from this approach, the proposed method directly optimizes LiDAR sensing and DSP parameters for downstream tasks. To investigate the optimization of LiDAR system parameters, we devise a realistic LiDAR simulation method that generates raw waveforms as input to a LiDAR DSP pipeline. We optimize LiDAR parameters for both 3D object detection IoU losses and depth error metrics by solving a nonlinear multi-objective optimization problem with a 0th-order stochastic algorithm. For automotive 3D object detection models, the proposed method outperforms manual expert tuning by 39.5% mean Average Precision (mAP).

Cited by 3

Accelerating hyperparameter optimization with a secretary

Víctor Muñoz, Carmen Ballester, Dorin Copaci et al. · Neurocomputing · 2025

Hyperparameter optimization (HPO) is a vital step in machine learning (ML) for enhancing model performance. However, the vast and complex nature of the search space makes HPO both challenging and resource-intensive. Automatic HPO methods have demonstrated their ability to efficiently explore high-dimensional hyperparameter spaces and identify optimal solutions, but training and evaluating the model for each set of hyperparameters is still computationally expensive. To further reduce the computational cost, we propose a novel strategy that wraps the HPO process and terminates it based on the sequence of hyperparameters evaluated. The algorithm is inspired by the classic secretary problem, with two additional variations to better adjust to the HPO process. We evaluated the algorithm using popular HPO samplers, including Random Search (RS), Tree-structured Parzen Estimator (TPE), Bayesian Optimization with Gaussian Processes (BOGP), Genetic Algorithms (GA), and Particle Swarm Optimization (PSO). Results indicate that the proposed algorithm accelerates the HPO process by an average of 34%, with only a minimal trade-off in solution quality of 8%. The algorithm is straightforward to implement, compatible with any HPO setup, and particularly effective in the early stages of optimization. This makes it a valuable tool for practitioners aiming to quickly identify promising hyperparameters or reducing the search space, significantly cutting down the time and computational resources required. • Novel strategy for accelerating the optimization process. • The secretary problem as a framework for hyperparameter optimization. • Analysis of the proposed strategy with the most common samplers.

Cited by 2Open access

Algorithm of ACO–SVR hyperparameter optimization.

C. Kishor Kumar Reddy (18078564), Vijaya Sindhoori Kaza (18078567), P. R. Anisha (18078570) et al. · Figshare · 2024

&lt;div&gt;&lt;p&gt;This research addresses the pressing challenge of intrusion detection and prevention in Wireless Sensor Networks (WSNs), offering an innovative and comprehensive approach. The research leverages Support Vector Regression (SVR) models to predict the number of barriers necessary for effective intrusion detection and prevention while optimising their strategic placement. The paper employs the Ant Colony Optimization (ACO) algorithm to enhance the precision of barrier placement and resource allocation. The integrated approach combines SVR predictive modelling with ACO-based optimisation, contributing to advancing adaptive security solutions for WSNs. Feature ranking highlights the critical influence of barrier count attributes, and regularisation techniques are applied to enhance model robustness. Importantly, the results reveal substantial percentage improvements in model accuracy metrics: a 4835.71% reduction in Mean Squared Error (MSE) for ACO-SVR1, an 862.08% improvement in Mean Absolute Error (MAE) for ACO-SVR1, and an 86.29% enhancement in R-squared (R&lt;sup&gt;2&lt;/sup&gt;) for ACO-SVR1. ACO-SVR2 has a 2202.85% reduction in MSE, a 733.98% improvement in MAE, and a 54.03% enhancement in R-squared. These considerable improvements verify the method’s effectiveness in enhancing WSNs, ensuring reliability and resilience in critical infrastructure. The paper concludes with a performance comparison and emphasises the remarkable efficacy of regularisation. It also underscores the practicality of precise barrier count estimation and optimised barrier placement, enhancing the security and resilience of WSNs against potential threats.&lt;/p&gt;&lt;/div&gt;

Cited by 0Open access

Hyperparameter Optimization in Machine Learning Algorithms

July Andrea Acero Lozada, Kevin Stiveen Rojas Ramirez · Universidad Industrial de Santander · 2023

En la actualidad, el Automated Machine Learning (AutoML) ha sido ampliamente aplicado debido al alto potencial benéfico que aporta a los distintos sectores de la industria, particularmente en la mejora de los flujos de trabajo, el rendimiento de los procesos y la efectividad empresarial. Dentro del AutoML es importante la adecuada elección de los valores de hiperparámetros mediante técnicas de optimización, ya que los algoritmos de ML dependen de los valores de hiperparámetros (HPs) elegidos debido a que estos influyen en el rendimiento de la máquina. Por ello, en esta investigación se diseña metodológicamente el proceso de construcción de los espacios de búsqueda y se propone una técnica para la optimización hiperparámetros de una máquina de Random Forest (RF) mediante la adaptación de la metaheurística de Particle Swarm Optimization (PSO). Esta técnica se utiliza para el análisis de conjunto de datos equilibrados y desequilibrados, mediante un proceso de benchmarking. Para validar el rendimiento del método, inicialmente se ejecutó el algoritmo de Random Forest sin aplicar la técnica de optimización, en donde se encontró la existencia de sobreajuste. Para reducir este comportamiento se establece una comparación entre PSO y técnicas de mayor usabilidad como Grid Search y Random Search a fin de analizar el comportamiento del modelo e identificar la técnica más efectiva en términos de funcionalidad y rendimiento. Los resultados demuestran que el modelo de clasificación Random Forest junto con la técnica de optimización de hiperparámetros Particle Swarm Optimization (PSO) mejoró la eficacia general del modelo, posibilitando obtener valores óptimos de hiperparámetros que mejoraron el rendimiento y el sobreajuste del modelo.

Cited by 0Open access

Hyperparameter Optimization for Unsupervised Outlier Detection

Yue Zhao, Leman Akoglu · arXiv (Cornell University) · 2022

Given an unsupervised outlier detection (OD) algorithm, how can we optimize its hyperparameter(s) (HP) on a new dataset, without any labels? In this work, we address this challenging hyperparameter optimization for unsupervised OD problem, and propose the first systematic approach called HPOD that is based on meta-learning. HPOD capitalizes on the prior performance of a large collection of HPs on existing OD benchmark datasets, and transfers this information to enable HP evaluation on a new dataset without labels. Moreover, HPOD adapts a prominent sampling paradigm to identify promising HPs efficiently. Extensive experiments show that HPOD works with both deep (e.g., Robust AutoEncoder) and shallow (e.g., Local Outlier Factor (LOF) and Isolation Forest (iForest)) OD algorithms on discrete and continuous HP spaces, and outperforms a wide range of baselines with on average 58% and 66% performance improvement over the default HPs of LOF and iForest.

Cited by 0Open access

Hyperparameter Optimization through Neural Network Partitioning

Bruno Mlodozeniec, Matthias Reisser, Christos Louizos · arXiv (Cornell University) · 2023

Well-tuned hyperparameters are crucial for obtaining good generalization behavior in neural networks. They can enforce appropriate inductive biases, regularize the model and improve performance -- especially in the presence of limited data. In this work, we propose a simple and efficient way for optimizing hyperparameters inspired by the marginal likelihood, an optimization objective that requires no validation data. Our method partitions the training data and a neural network model into $K$ data shards and parameter partitions, respectively. Each partition is associated with and optimized only on specific data shards. Combining these partitions into subnetworks allows us to define the ``out-of-training-sample" loss of a subnetwork, i.e., the loss on data shards unseen by the subnetwork, as the objective for hyperparameter optimization. We demonstrate that we can apply this objective to optimize a variety of different hyperparameters in a single training run while being significantly computationally cheaper than alternative methods aiming to optimize the marginal likelihood for neural networks. Lastly, we also focus on optimizing hyperparameters in federated learning, where retraining and cross-validation are particularly challenging.

Cited by 0Open access

Bayesian Hyperparameter Optimization for Ensemble Learning

Julien-Charles Lévesque, Christian Gagné, Robert Sabourin · arXiv (Cornell University) · 2016

In this paper, we bridge the gap between hyperparameter optimization and ensemble learning by performing Bayesian optimization of an ensemble with regards to its hyperparameters. Our method consists in building a fixed-size ensemble, optimizing the configuration of one classifier of the ensemble at each iteration of the hyperparameter optimization algorithm, taking into consideration the interaction with the other models when evaluating potential performances. We also consider the case where the ensemble is to be reconstructed at the end of the hyperparameter optimization phase, through a greedy selection over the pool of models generated during the optimization. We study the performance of our proposed method on three different hyperparameter spaces, showing that our approach is better than both the best single model and a greedy ensemble construction over the models produced by a standard Bayesian optimization.

Cited by 35Open access

Automatic hyperparameter optimization for machine learning

Xavier Tan · DR-NTU (Nanyang Technological University) · 2020

This project focuses on the concept of hyperparameters in a Machine Learning classifi- cation problem. Hyperparameters are parameter values that have direct control over the behaviour of the Machine Learning classification model. Automatic hyperparameter op- timization is an active area of research which aims to facilitate the exhaustive process of manually tuning them by hand. In recent years, the success of the bayesian optimization model in the sequential model-based optimization approach shed light and direction for the future of hyperparameter optimization. It led to the development of many automatic hyperparameter optimization frameworks such as Hyperopt, Scikit-optimize and Optuna, to name a few. Each differs in the underlying optimization algorithms and performance. A considerable amount of effort is required to understand the complex and rigid struc- ture of the framework, and they are not easily reproducible onto other practical Machine Learning problems.&#13;\nThis project aims to address those problems through the development of a unified func- tion combining Hyperopt, Scikit-optimize and Optuna, offering more flexibility, improved usability and visual aids. Experiments conducted demonstrated that the program devel- oped had consistently produced a model that outperforms an untuned classification model and that Hyperopt is superior in terms of accuracy score and speed. However, an ob- servation from the results was that the program hadn’t accounted for several factors which could have impeded with the performance of the random forest classifier. Hence, as future works, functional ANOVA can be incorporated into the program to address the limitations of the program.

Cited by 0

Efficient hyperparameter tuning using Bayesian optimization

Theckel Joy, Tinu · Own your potential (DEAKIN) · 2019

\n\t\t\t\t\tThis thesis addresses many open challenges in hyperparameter tuning of machine learning algorithms. The thesis develops efficient Bayesian optimization frameworks for hyperparameter tuning by utilizing different techniques like transfer learning, parallel computing, and domain-specific prior knowledge induction.\n\t\t\t\t

Cited by 0