preprint · arXiv (Cornell University) · 2018

Automatic Gradient Boosting

Janek Thomas, Stefan Coors, Bernd Bischl

Cited by 7Open access

Abstract

Automatic machine learning performs predictive modeling with high performing machine learning tools without human interference. This is achieved by making machine learning applications parameter-free, i.e. only a dataset is provided while the complete model selection and model building process is handled internally through (often meta) optimization. Projects like Auto-WEKA and auto-sklearn aim to solve the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem resulting in huge configuration spaces. However, for most real-world applications, the optimization over only a few different key learning algorithms can not only be sufficient, but also potentially beneficial. The latter becomes apparent when one considers that models have to be validated, explained, deployed and maintained. Here, less complex model are often preferred, for validation or efficiency reasons, or even a strict requirement. Automatic gradient boosting simplifies this idea one step further, using only gradient boosting as a single learning algorithm in combination with model-based hyperparameter tuning, threshold optimization and encoding of categorical features. We introduce this general framework as well as a concrete implementation called autoxgboost. It is compared to current AutoML projects on 16 datasets and despite its simplicity is able to achieve comparable results on about half of the datasets as well as performing best on two.

Cite this paper

APA 7

Thomas, J., Coors, S., & Bischl, B. (2018). Automatic Gradient Boosting. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1807.03873

MLA 9

Thomas, Janek, et al. “Automatic Gradient Boosting.” arXiv (Cornell University), 2018, https://doi.org/10.48550/arxiv.1807.03873.

Chicago

Thomas, Janek, Stefan Coors, and Bernd Bischl. 2018. “Automatic Gradient Boosting.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1807.03873.

Harvard

Thomas, J., Coors, S. and Bischl, B. (2018) ‘Automatic Gradient Boosting’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.1807.03873.

BibTeX

@misc{thomas2018automatic,
  title = {{Automatic Gradient Boosting}},
  author = {Thomas, Janek and Coors, Stefan and Bischl, Bernd},
  journal = {arXiv (Cornell University)},
  year = {2018},
  doi = {10.48550/arxiv.1807.03873}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.