Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “Gradient boosting”

Gradient Boosting

Zhiyuan Wang, Sayed Ameenuddin Irfan, Christopher Teoh et al. · BENTHAM SCIENCE PUBLISHERS eBooks · 2023

In this chapter, we explore gradient boosting, a powerful ensemble machine learning method, for both regression and classification tasks. With a focus on accessibility, we minimize abstract mathematical theories and instead emphasize two concrete numerical examples with small datasets related to predicting house sale prices and ease of selling houses in the property market. By providing a step-by-step walkthrough, we illuminate the inner workings of gradient boosting and offer sample codes and comparisons to the gradient boosting models available in the scikit-learn library. Upon completing this chapter, readers will possess a comprehensive understanding of gradient boosting's mechanics, its connection to the implementation and performance of the algorithm, and be well-prepared to apply it in real-world projects.

Cited by 1

Gradient boosting

University of Oulu Repository (University of Oulu) · 2021

Cited by 0

Gradient-Free Gradient Boosting

Tino Werner · Carl von Ossiezky University of Oldenburg · 2020

Motivated by applications in fraud detection, this dissertation is concerned about model selection in predictive models where the correct ranking of observations has to be predicted. For this, the thesis starts by proving the asymptotic linearity of a whole family of regularized M-estimators which covers for example the Lasso. With the algorithm ''SingBoost'' developed in this dissertation, we succeed in providing a Gradient Boosting algorithm as an extension of L2-Boosting, even though the loss function is non-continuous. We prove: This algorithm has analogous consistency properties as L2-Boosting. As to stable model selection, we develop a loss-based Stability Selection. In combination with this Stability Selection, SingBoost strictly improves the performance for the hard ranking
\nproblem on simulated data. The loss-based Stability Selection that we provide is universally applicable, i.e., for arbitrary loss
\nfunctions.

Cited by 2Open access

gfboost: Gradient-Free Gradient Boosting

Tino Werner · 2020

Implementation of routines of the author's PhD thesis on gradient-free Gradient Boosting (Werner, Tino (2020) "Gradient-Free Gradient Boosting", URL '&lt;<a href="https://oops.uni-oldenburg.de/id/eprint/4290" target="_top">https://oops.uni-oldenburg.de/id/eprint/4290</a>&gt;').

Cited by 1Open access

bst: Gradient Boosting

Zhu Wang · 2011

Functional gradient descent algorithm for a variety of convex and non-convex loss functions, for both classical and robust regression and classification problems. See Wang (2011) &lt;<a href="https://doi.org/10.2202%2F1557-4679.1304" target="_top">doi:10.2202/1557-4679.1304</a>&gt;, Wang (2012) &lt;<a href="https://doi.org/10.3414%2FME11-02-0020" target="_top">doi:10.3414/ME11-02-0020</a>&gt;, Wang (2018) &lt;<a href="https://doi.org/10.1080%2F10618600.2018.1424635" target="_top">doi:10.1080/10618600.2018.1424635</a>&gt;, Wang (2018) &lt;<a href="https://doi.org/10.1214%2F18-EJS1404" target="_top">doi:10.1214/18-EJS1404</a>&gt;.

Cited by 1Open access

Gradient Boosting Dataset

Rosalía Palomino‐Cabrera · Zenodo (CERN European Organization for Nuclear Research) · 2021

Dataset of the samples used for the Gradient Boosting model construction. Each folder contains the original fasta file of the genome, the assigned coverage of each molecule and the synthetic reads used to reconstruct the genome.

Cited by 0Open access

Stochastic Gradient Boosting ∗

Brian Kriegler, Richard Berk · 2010

In many metropolitan areas, efforts are made to count the homeless to ensure proper provision of social services. Some areas are very large, which makes spatial sampling a viable alternative to an enumeration of the entire terrain. Counts are observed in sampled regions but must be imputed in unvisited areas. Along with the imputation process, the costs of underestimating and overestimating may be different. For example, if precise estimation in areas with large homeless counts is critical, then underestimation should be penalized more than overestimation in the loss function. We analyze data from the 2004-2005 Los Angeles County homeless study using an augmentation of L1 stochastic gradient boosting that can weight overestimates and underestimates asymmetrically. We discuss our choice to utilize stochastic gradient boosting over other function estimation procedures. In-sample fitted and out-of-sample imputed values, as well as relationships between the response and predictors, are analyzed for various cost functions. We gratefully acknowledge numerous helpful discussions with Greg Ridgeway. The

Cited by 0

Online Gradient Boosting

Alina Beygelzimer, Elad Hazan, Satyen Kale et al. · arXiv (Cornell University) · 2015

We extend the theory of boosting for regression problems to the online learning setting. Generalizing from the batch setting for boosting, the notion of a weak learning algorithm is modeled as an online learning algorithm with linear loss functions that competes with a base class of regression functions, while a strong learning algorithm is an online learning algorithm with convex loss functions that competes with a larger class of regression functions. Our main result is an online gradient boosting algorithm which converts a weak online learning algorithm into a strong one where the larger class of functions is the linear span of the base class. We also give a simpler boosting algorithm that converts a weak online learning algorithm into a strong one where the larger class of functions is the convex hull of the base class, and prove its optimality.

Cited by 33Open access

Condensed-gradient boosting

Seyedsaman Emami, Gonzalo Martínez-Muñoz · International Journal of Machine Learning and Cybernetics · 2024

Abstract This paper presents a computationally efficient variant of Gradient Boosting (GB) for multi-class classification and multi-output regression tasks. Standard GB uses a 1-vs-all strategy for classification tasks with more than two classes. This strategy entails that one tree per class and iteration has to be trained. In this work, we propose the use of multi-output regressors as base models to handle the multi-class problem as a single task. In addition, the proposed modification allows the model to learn multi-output regression problems. An extensive comparison with other multi-output based Gradient Boosting methods is carried out in terms of generalization and computational efficiency. The proposed method showed the best trade-off between generalization ability and training and prediction speeds. Furthermore, an analysis of space and time complexity was undertaken.

Cited by 18Open access

Automatic Gradient Boosting

Janek Thomas, Stefan Coors, Bernd Bischl · arXiv (Cornell University) · 2018

Automatic machine learning performs predictive modeling with high performing machine learning tools without human interference. This is achieved by making machine learning applications parameter-free, i.e. only a dataset is provided while the complete model selection and model building process is handled internally through (often meta) optimization. Projects like Auto-WEKA and auto-sklearn aim to solve the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem resulting in huge configuration spaces. However, for most real-world applications, the optimization over only a few different key learning algorithms can not only be sufficient, but also potentially beneficial. The latter becomes apparent when one considers that models have to be validated, explained, deployed and maintained. Here, less complex model are often preferred, for validation or efficiency reasons, or even a strict requirement. Automatic gradient boosting simplifies this idea one step further, using only gradient boosting as a single learning algorithm in combination with model-based hyperparameter tuning, threshold optimization and encoding of categorical features. We introduce this general framework as well as a concrete implementation called autoxgboost. It is compared to current AutoML projects on 16 datasets and despite its simplicity is able to achieve comparable results on about half of the datasets as well as performing best on two.

Cited by 7Open access

Accelerated Gradient Boosting

Biau, Gérard, Cadre, Benoît, Rouvìère, Laurent · HAL (Le Centre pour la Communication Scientifique Directe) · 2019

Cited by 0Open access

Infinitesimal gradient boosting

Clément Dombry, Jean‐Jil Duchamps · arXiv (Cornell University) · 2021

We define infinitesimal gradient boosting as a limit of the popular tree-based gradient boosting algorithm from machine learning. The limit is considered in the vanishing-learning-rate asymptotic, that is when the learning rate tends to zero and the number of gradient trees is rescaled accordingly. For this purpose, we introduce a new class of randomized regression trees bridging totally randomized trees and Extra Trees and using a softmax distribution for binary splitting. Our main result is the convergence of the associated stochastic algorithm and the characterization of the limiting procedure as the unique solution of a nonlinear ordinary differential equation in a infinite dimensional function space. Infinitesimal gradient boosting defines a smooth path in the space of continuous functions along which the training error decreases, the residuals remain centered and the total variation is well controlled.

Cited by 0Open access

Condensed Gradient Boosting

Seyedsaman Emami, Gonzalo Martínez-Muñoz · arXiv (Cornell University) · 2022

This paper presents a computationally efficient variant of gradient boosting for multi-class classification and multi-output regression tasks. Standard gradient boosting uses a 1-vs-all strategy for classifications tasks with more than two classes. This strategy translates in that one tree per class and iteration has to be trained. In this work, we propose the use of multi-output regressors as base models to handle the multi-class problem as a single task. In addition, the proposed modification allows the model to learn multi-output regression problems. An extensive comparison with other multi-ouptut based gradient boosting methods is carried out in terms of generalization and computational efficiency. The proposed method showed the best trade-off between generalization ability and training and predictions speeds.

Cited by 0Open access

Distributional Gradient Boosting Machines

Alexander März, Thomas Kneib · arXiv (Cornell University) · 2022

We present a unified probabilistic gradient boosting framework for regression tasks that models and predicts the entire conditional distribution of a univariate response variable as a function of covariates. Our likelihood-based approach allows us to either model all conditional moments of a parametric distribution, or to approximate the conditional cumulative distribution function via Normalizing Flows. As underlying computational backbones, our framework is based on XGBoost and LightGBM. Modelling and predicting the entire conditional distribution greatly enhances existing tree-based gradient boosting implementations, as it allows to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived. Empirical results show that our framework achieves state-of-the-art forecast accuracy.

Cited by 1Open access

Accelerating Gradient Boosting Machine

Karimireddy, Sai Praneeth Reddy, Lu, Haihao, Ponomareva, Natalia et al. · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · 2020

Cited by 0Open access