Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.
About 10,000 papers for “kernel selection”
Lior Wolf And, Lior Wolf, Amnon Shashua · 2003
We address the problem of using a kernel spectral criterion function for feature selection. A feature selection paradigm using spectral properties of the affinity matrix of the input data was recently introduced in [11] and which leads to a bilinear interaction between entries of the data sample.
Cited by 0
Guang Dai, Dit-yan Yeung · 2007
Existing semi-supervised learning methods are mostly based on either the cluster assumption or the manifold assumption. In this paper, we propose an integrated regularization framework for semi-supervised kernel machines by incorporating both the cluster assumption and the manifold assumption. Moreover, it supports kernel learning in the form of kernel selection. The optimization problem involves joint optimization over all the labeled and unlabeled data points, a convex set of basic kernels, and a discrete space of unknown labels for the unlabeled data. When the manifold assumption is incorporated, graph Laplacian kernels are used as the basic kernels for learning an optimal convex combination of graph Laplacian kernels. Comparison with related methods on the USPS data set shows very promising results. 1.
Cited by 0
Seung-Jean Kim, Alessandro Magnani, Stephen Boyd · 2006
In Kernel Fisher discriminant analysis (KFDA), we carry out Fisher linear discriminant analysis in a high dimensional feature space defined implicitly by a kernel. The performance of KFDA depends on the choice of the kernel; in this paper, we consider the problem of finding the optimal kernel, over a given convex set of kernels. We show that this optimal kernel selection problem can be reformulated as a tractable convex optimization problem which interior-point methods can solve globally and efficiently. The kernel selection method is demonstrated with some UCI machine learning benchmark examples.
Cited by 147
H. Halconruy, N. Marie · Mathematical Methods of Statistics · 2020
In the regression model $$Y=b(X)+\sigma(X)\varepsilon$$ , where $$X$$ has a density $$f$$ , this paper deals with an oracle inequality for an estimator of $$bf$$ , involving a kernel in the sense of Lerasle et al. [13], selected via the PCO method. In addition to the bandwidth selection for kernel-based estimators already studied in Lacour et al. [12] and Comte and Marie [3], the dimension selection for anisotropic projection estimators of $$f$$ and $$bf$$ is covered.
Cited by 3Open access
Mohamed Gebril, Abdollah Homaifar · 2013
Machine Learning is considered as a subfield of Artificial Intelligence and it is concerned with the development of techniques and methods which enable the computer to learn. In simple terms development of algorithms which enable the machine to learn and perform tasks and activities. Machine learning (ML) overlaps with statistics in many ways in engineering problems. Predictive models are ML
Cited by 0
Di You · OhioLink ETD Center (Ohio Library and Information Network) · 2011
Cited by 2Open access
Yong Liu, Shizhong Liao, Hua Zhang et al. · IEEE Transactions on Cybernetics · 2019
Model selection is one of the fundamental problems in kernel-based algorithms, which is commonly done by minimizing an estimation of generalization error. The notion of stability and cross-validation (CV) error of learning machines consists of two widely used tools for analyzing the generalization performance. However, there are some disadvantages to both tools when applied for model selection: 1) the stability of learning machines is not practical due to the difficulty of the estimation of its specific value and 2) the CV-based estimate of generalization error usually has a relatively high variance, so it is prone to overfitting. To overcome these two limitations, we present a novel notion of kernel stability (KS) for deriving the generalization error bounds and variance bounds of CV and provide an effective approach to the application of KS for practical model selection. Unlike the existing notions of stability of the learning machine, KS is defined on the kernel matrix; hence, it can avoid the difficulty of the estimation of its value. We manifest the relationship between the KS and the popular uniform stability of the learning algorithm, and further propose several KS-based generalization error bounds and variance bounds of CV. By minimizing the proposed bounds, we present two novel KS-based criteria that can ensure good performance. Finally, we empirically analyze the performance of the proposed criteria on many benchmark data, which demonstrates that our KS-based criteria are sound and effective.
Cited by 6
Seung-jean Kim, Alessandro Magnani, Stephen Boyd · 2013
In Kernel Fisher discriminant analysis (KFDA), we carry out Fisher linear discriminant analysis in a high dimensional feature space defined implicitly by a kernel. The performance of KFDA depends on the choice of the kernel; in this paper, we consider the problem of finding the optimal kernel, over a given convex set of kernels. We show that this optimal kernel selection problem can be reformulated as a tractable convex optimization problem which interior-point methods can solve globally and efficiently. The kernel selection method is demonstrated with some UCI machine learning benchmark examples. 1.
Cited by 0
POTHIN, Jean-Baptiste, RICHARD, Cédric · OpenGrey (Institut de l'Information Scientifique et Technique) · 2007
Dans le foisonnant domaine des méthodes à noyaux, force est de constater aujourd hui que les efforts de recherche ont été essentiellement portés sur les applications, délaissant quelque peu l étude des noyaux eux-mêmes. Cet état de fait est d autant plus surprenant que ces derniers conditionnent largement l efficacité des traitements, par leur aptitude à révéler les similarités existant au sein des données traitées. Le principal objectif de cette thèse est de proposer une méthodologie pour l élaboration d un noyau adapté à un problème et à des données. La première partie de ce manuscrit s intéresse au problème d apprentissage d un noyau par ajustement de ses paramètres libres. Nous étudions en particulier les propriétés statistiques des critères d alignement et de polarité, et proposons un algorithme glouton pour l optimisation d une combinaison linéaire de noyaux. Dans la seconde partie, nous étudions le problème d optimisation de la représentation des données. Un algorithme est proposé pour transformer linéairement les données selon le critère d alignement. Afin de lutter contre la malédiction de la dimensionnalité, nous étudions une seconde approche basée sur l optimisation de la métrique et l appliquons avec succès au problème de classification de textes. La dernière partie concerne la prise en compte d a priori modélisé sous la forme d ellipsoïdes, inspiré de l apprentissage d un SVM, nous développons un algorithme supervisé capable de classer efficacement des ellipsoïdes
Cited by 0Open access
Association for Artificial Intelligence 2021, Liao, Shizhong, Wen, Ji-Rong et al. · Underline Science Inc. · 2021
Regret bounds of online kernel selection in a finite kernel set have been well studied, having at least an order O( √ NT) of magnitude after T rounds, where N is the number of candidate kernels. But it is still an unsolved problem to achieve sublinear regret bounds of online kernel selection in a continuous kernel space under different learning frameworks. In this paper, to represent different learning frameworks of online kernel selection, we divide online kernel selection approaches in a continuous kernel space into two categories according to the order of selection and training at each round. Then we construct a surrogate hypothesis space that contains all the candidate kernels with bounded norms and inner products, representing the continuously varying hypothesis space. Finally, we decompose the regrets of the proposed online kernel selection categories into different types of instantaneous regrets in the surrogate hypothesis space, and derive optimal regret bounds of order O( √ T) of magnitude under mild assumptions, independent of the cardinality of the continuous kernel space. Empirical studies verified the correctness of the theoretical regret analyses.
Cited by 0Open access
WangChao, LiBo, LiuHuan et al. · Research Square · 2022
Abstract Kernel regression is widely used in biology and economy, because it is more adaptable to complex laws than linear regression, and it has better interpretability than many methods in deep learning. In highdimensional area, l1-norm penalization is a common method for variable selection, which may be derived from the excellent performance of the lasso algorithm. Although it seems natural to generalize from consistency in variable selection to consistency in kernel selection, there are still many details that need to be taken seriously, e.g. the lower eigenvalue condition. The consistence condition of kernel selection in l1-norm regular linear kernel regression is given, including the prediction error. In simulation study, the consistency of different levels of λn and dimension of features is carefully checked. Finally, the kernel selection method was applied to high-risk area exploration of Covid-19, with the dataset provided by the US Centers for Disease Control and Prevention(CDC). Declarations of interest:The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Cited by 0Open access
Jian Li, Yong Liu, Hailun Lin et al. · 2017
Kernel selection is a fundamental problem of kernel methods. Existing measures for kernel selection either provide less theoretical guarantee or have high computational complexity. In this paper, we propose a novel kernel selection criterion based on a newly defined spectral measure of a kernel matrix, with sound theoretical foundation and high computational efficiency. We first show that the spectral measure can be used to derive generalization bounds for some kernel-based algorithms. By minimizing the derived generalization bounds, we propose the kernel selection criterion with spectral measure. Moreover, we demonstrate that the popular minimum graph cut and maximum mean discrepancy are two special cases of the proposed criterion. Experimental results on lots of data sets show that our proposed criterion can not only give the comparable results as the state-of-the-art criterion, but also significantly improve the efficiency.
Cited by 7Open access
Paulo Diniz, Jonathas Ferreira, Marcele Kuhfuss et al. · IEEE RESOURCE CENTERS · 2024
Cited by 0Open access
Alireza Asvadi, Guillaume Dardenne, Aziliz Guezou‐Philippe et al. · HAL (Le Centre pour la Communication Scientifique Directe) · 2019
Proceedings en ligne dans HALhttps://hal.archives-ouvertes.fr/hal-02183893
Cited by 1Open access
A.J. Smith, Kate Smith‐Miles · IGI Global eBooks · 2011
The most critical component of kernel based learning algorithms is the choice of an appropriate kernel and its optimal parameters. In this paper we propose a rule based meta-learning approach for automatic radial basis function (rbf) kernel and its parameter selection for Support Vector Machine (SVM) classification. First, the best parameter selection is considered on the basis of prior information of the data with the help of Maximum Likelihood (ML) method and Nelder-Mead (N-M) simplex method. Then the new rule based meta-learning approach is constructed and tested on different sizes of 112 datasets with binary class as well as multi class classification problems. We observe that our rule based methodology provides significant improvement of computational time as well as accuracy in some specific cases.
Cited by 0
Shawkat Ali, K.A. Smith · Figshare · 2004
Kernel is the heart of kernel based learning. To choose an appropriate parameter for a specific kernel is an important research issue in the data mining area. In this paper, we propose an automatic parameter selection approach for polynomial kernel. The algorithm is tested on support vector machines (SVM). The parameter selection is considered on the basis of prior information of the data distribution and Bayesian inference. The new approach is tested on different sizes of benchmark datasets with binary class problems as well as multi class classification problems.
Cited by 47Open access
X. S. Jiang, Yucong Dai, Yongkai Wu · 2023
With the prevalence of machine learning in many high-stakes decision-making processes, e.g., hiring and admission, it is important to take fairness into consideration when practitioners design and deploy machine learning models. Although many approaches have been developed for fair machine learning, most of them focus on classification. In this paper, we target a notable but under-explored task, selection, where the number of selected individuals cannot exceed a pre-defined budget, such as employee hiring or university admission with limited positions or capabilities. In the selection task, existing fairness notions designed for classification are not suitable. In particular, our experimental results show that the selection models subject to common fairness notions may still make biased predictions against the underrepresented group. Hence, we propose a novel fairness notion, Selection Parity, which captures the demographic diversity among the selected groups in this restricted selection problem. Since the selection of qualified individuals with a fixed budget is non-differentiable, existing fairness regularization terms cannot be directly integrated with the selection task. To close the gap, we develop a novel in-processing framework named Fair Selection with the Differentiable Distribution Difference constraint (FS-DD), which incorporates a differentiable constraint into the training process and produces fair decisions for selection problems. Our theoretical analysis shows that common fairness metrics are bounded by the proposed Distribution Difference measurement. In other words, the FS-DD framework can guarantee fairness with regard to the common existing fairness metrics. We evaluate the performance of our method as well as several baselines on four real-world datasets. The experimental results demonstrate that the proposed method achieves fairness in various selection settings. In addition, the proposed method has a better fairness-accuracy trade-off compared with existing baseline methods.
Cited by 4
Hidenori Okumura, Kanta Naito · Journal of nonparametric statistics · 2006
In nonparametric binomial regression, the weighted kernel estimator of the regression function and its efficient bias-adjusted version have been proposed by Okumura and Naito (2004) with consideration to differences of variances of observed response proportions at covariates. The aim of this article is to propose an effective data-based method for bandwidth selection of the bias-adjusted estimator. The proposed method is developed through three steps: the plug-in method by Ruppert et al. (1995), a scale adjustment suggested by Yang and Tschernig (1999), and an effective use of the approach discussed by Grizzel et al. (1969) for the rule-of-thumb part. Theoretical considerations on the asymptotic performance of the selected bandwidth are given under the situation where the numbers of covariates and responses observed at each covariate increase. (References: Okumura, H. and Naito, K., 2004, The weighted estimators in nonparametric binomial regression. Journal of Nonparametric Statistics, 16, 39–62; Ruppert, D., Sheather, S.J. and Wand, D.W., 1995, An effective band width selector for local least squares regression. Journal of the American Statistical Association, 90, 1257–1270; Yang, L. and Tschernig, R., 1999, Multivariate bandwidth selection for local linear regression. Journal of the Royal Statistical Society, 61, 793–815; Grizzel, J.E., Starmer, C.F. and Koch, G.G., 1969, Analysis of growth and dose-response curves. Biometrics, 25, 489–504.)
Cited by 6
Xiaoyu Qiu, Kai Kang, Huaxiang Zhang · 2008
How to choose the optimal parameter is crucial for the kernel method, because kernel parameters perform significantly on the kernel method. In this paper, a novel approach is proposed to choose the kernel parameter for the kernel nearest-neighbor classifier (KNN). The values of the kernel parameter are computed through optimizing an object function designed for measuring the classification reliability of KNN. We test our approach on both artificial and real-word datasets, and the preliminary results demonstrate that our approach provides a practical solution.
Cited by 6
Peter Exterkate · Computational Statistics & Data Analysis · 2013
Cited by 60