article · RECERCAT (Consorci de Serveis Universitaris de Catalunya)

Classifier selection with permutation tests

Arias Vicente, Marta, Arratia Quesada, Argimiro Alejandro, Duarte López, Ariel

Cited by 0Open access

Abstract

This work presents a content-based recommender system for machine learning classifier algorithms. Given a new data set, a recommendation of what classifier is likely to perform best is made based on classifier performance over similar known data sets. This similarity is measured according to a data set characterization that includes several state-of-the-art metrics taking into account physical structure, statistics, and information theory. A novelty with respect to prior work is the use of a robust approach based on permutation tests to directly assess whether a given learning algorithm is able to exploit the attributes in a data set to predict class labels, and compare it to the more commonly used F-score metric for evaluating classifier performance. To evaluate our approach, we have conducted an extensive experimentation including 8 of the main machine learning classification methods with varying configurations and 65 binary data sets, leading to over 2331 experiments. Our results show that using the information from the permutation test clearly improves the quality of the recommendations.

Cite this paper

APA 7

Arias Vicente, M., Arratia Quesada, A. A., & Duarte López, A. (n.d.). Classifier selection with permutation tests. RECERCAT (Consorci de Serveis Universitaris de Catalunya).

MLA 9

Arias Vicente, Marta, et al. “Classifier selection with permutation tests.” RECERCAT (Consorci de Serveis Universitaris de Catalunya).

Chicago

Arias Vicente, Marta, Argimiro Alejandro Arratia Quesada, and Ariel Duarte López. n.d.. “Classifier selection with permutation tests.” RECERCAT (Consorci de Serveis Universitaris de Catalunya).

Harvard

Arias Vicente, M., Arratia Quesada, A.A. and Duarte López, A. (n.d.) ‘Classifier selection with permutation tests’, RECERCAT (Consorci de Serveis Universitaris de Catalunya).

BibTeX

@article{ariasvicenteclassifier,
  title = {{Classifier selection with permutation tests}},
  author = {Arias Vicente, Marta and Arratia Quesada, Argimiro Alejandro and Duarte López, Ariel},
  journal = {RECERCAT (Consorci de Serveis Universitaris de Catalunya)}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.