Martin L. Hazelton · Australian & New Zealand Journal of Statistics · 2003
Summary This paper considers the problem of selecting optimal bandwidths for variable (sample‐point adaptive) kernel density estimation. A data‐driven variable bandwidth selector is proposed, based on the idea of approximating the log‐bandwidth function by a cubic spline. This cubic spline is optimized with respect to a cross‐validation criterion. The proposed method can be interpreted as a selector for either integrated squared error (ISE) or mean integrated squared error (MISE) optimal bandwidths. This leads to reflection upon some of the differences between ISE and MISE as error criteria for variable kernel estimation. Results from simulation studies indicate that the proposed method outperforms a fixed kernel estimator (in terms of ISE) when the target density has a combination of sharp modes and regions of smooth undulation. Moreover, some detailed data analyses suggest that the gains in ISE may understate the improvements in visual appeal obtained using the proposed variable kernel estimator. These numerical studies also show that the proposed estimator outperforms existing variable kernel density estimators implemented using piecewise constant bandwidth functions.
Cited by 33
Efrén Cruz Cortés · 2017
Cited by 0
Cruz Cortes, Efren · Deep Blue (University of Michigan) · 2017
Nonparametric density estimation is a common and important task in many problems in machine learning. It consists in estimating a density function from available observations without making parametric assumptions on the generating distribution. Kernel means are nonparametric estimators composed of the average of simple functions, called kernels, centered at each data point. This work studies some relatives of these kernel means with structural similarity but which assign different weights to each kernel unit in order to attain certain desired characteristics. In particular, we present a sparse kernel mean estimator and a consistent kernel density estimator with fixed bandwidth parameter. First, regarding kernel means, we study the kernel density estimator (KDE) and the kernel mean embedding. These are frequently used to represent probability distributions, unfortunately, they face scalability issues. A single point evaluation of the kernel density estimator, for example, requires a computation time linear in the training sample size. To address this challenge, we present a method to efficiently construct a sparse approximation of a kernel mean. We do so by first establishing an incoherence-based bound on the approximation error. We then observe that, for any kernel with constant norm (which includes all translation invariant kernels), the bound can be efficiently minimized by solving the k-center problem. The outcome is a linear time construction of a sparse kernel mean, which also lends itself naturally to an automatic sparsity selection scheme. We demonstrate the computational gains of our method by looking at several benchmark data sets, as well as three applications involving kernel means: Euclidean embedding of distributions, class proportion estimation, and clustering using the mean-shift algorithm. Second we address the bandwidth selection problem in kernel density estimation. Consistency of the KDE requires that the kernel bandwidth tends to zero as the sample size grows. In this work, we investigate the question of whether consistency is still possible when the bandwidth is fixed, if we consider a more general class of weighted KDEs. To answer this question in the affirmative, we introduce the fixed-bandwidth KDE (fbKDE), obtained by solving a quadratic program, that consistently estimates any continuous square-integrable density. Rates of convergence are also established for the fbKDE for radial kernels and the box kernel under appropriate smoothness assumptions. Furthermore, in a simulation study we demonstrate that the fbKDE compares favorably to the standard KDE and the previously proposed variable bandwidth KDE.
Cited by 0Open access
Pap Er Id · 2007
In this paper, we formulate kernel density estimation for color invariant images. Although color invariant images are robust against changing illumination, they become unstable for noisy data.
Cited by 0
Th. Gevers, F. Aldershoff · 2007
Therefore, in this paper, we formulate kernel density estimation robustness against noisy data, and robustness against changing illumination. To achieve this, computational methods are presented to measure colour constant gradients. Further, a model is given to estimate the amount of sensor noise through these color constant gradients. The obtained uncertainty is subsequently used as a weighting term in the density estimation process.
Cited by 0
Christiaan Van Der Walt, Etienne Barnard · Proceedings of the AAAI Conference on Artificial Intelligence · 2017
Estimating the joint probability density function of a dataset is a central task in many machine learning applications. In this work we address the fundamental problem of kernel bandwidth estimation for variable kernel density estimation in high-dimensional feature spaces. We derive a variable kernel bandwidth estimator by minimizing the leave-one-out entropy objective function and show that this estimator is capable of performing estimation in high-dimensional feature spaces with great success. We compare the performance of this estimator to state-of-the art maximum-likelihood estimators on a number of representative high-dimensional machine learning tasks and show that the newly introduced minimum leave-one-out entropy estimator performs optimally on a number of high-dimensional datasets considered.
Cited by 8Open access
Al-haj Ebrahem, Mohammed Hassan; Department of Statistics, Yarmouk University, Al-Momani, Noor, Eidous, Omar; Prof. · Università del Salento · 2021
Cited by 0
Zhen Zhang, Yanning Zhang · Neurocomputing · 2014
Cited by 9
Mark Brewer · 2000
A new procedure is proposed for deriving variable bandwidths in univariate kernel density estimation. Rather than concentrate on minimising some criterion based upon the mean integrated square error (MISE) or related quantity, we build models for the data and use exact calculation or sampling methods to make inferences about the bandwidths as appropriate. These Bayesian models are based on a cross-validated representation of the data; it is noted that they allow for bandwidth selection which is flexible in terms of the amount of smoothing required and can be tailored to suit specific applications. One model in particular, which introduces direct dependencies between bandwidths of neighbouring points, is seen to produce adaptive estimates which are reliably smooth in low density areas while still allowing for different levels of smoothing in high density areas. Key words: kernel density estimation, Markov chain Monte Carlo, cross-validation, variable bandwidth. 1 Introduction This pa...
Cited by 0
Shou, Wenhao, Gupta, Sat, Khalil, Sadia · Figshare · 2025
Auxiliary information is widely used in various studies to improve the precision of estimation. In this article, we extend the application of auxiliary information within the context of randomized response techniques (RRT), building upon the prior research on kernel density estimation (KDE) under additive RRT models. Inspired by Mostafa and Ahmad (2019), we proposed a kernel density estimator that incorporates an auxiliary variable to enhance the accuracy of estimating the distribution of a sensitive variable. Extensive simulations are conducted to evaluate the performance of this proposed methodology, highlighting the advantages of utilizing auxiliary information and the impact of factors such as noise levels, sample size, and the correlation between the auxiliary variable and the sensitive variable.
Cited by 0Open access
Shou, Wenhao, Gupta, Sat, Khalil, Sadia · Figshare · 2025
Auxiliary information is widely used in various studies to improve the precision of estimation. In this article, we extend the application of auxiliary information within the context of randomized response techniques (RRT), building upon the prior research on kernel density estimation (KDE) under additive RRT models. Inspired by Mostafa and Ahmad (2019), we proposed a kernel density estimator that incorporates an auxiliary variable to enhance the accuracy of estimating the distribution of a sensitive variable. Extensive simulations are conducted to evaluate the performance of this proposed methodology, highlighting the advantages of utilizing auxiliary information and the impact of factors such as noise levels, sample size, and the correlation between the auxiliary variable and the sensitive variable.
Cited by 0Open access
А. V. Lapko, В. А. Лапко, A. V. Bakhtina · Izmeritel`naya Tekhnika · 2021
Cited by 1
Jinxin Wang, Shenglei Zhao, Enyuan Wang et al. · Sensors · 2023
Incipient fault detection in a hydraulic system is a challenge in the condition monitoring community. Existing research mainly monitors abnormal working conditions in hydraulic systems by separately detecting the key working parameter, which often causes a high miss warning rate for incipient faults due to the oversight of parameter dependence. A principal component analysis provides an effective method for incipient fault detection by taking the correlation of multiple parameters into consideration, but this technique assumes the systems are Gaussian-distributed, making it invalid for a dynamic non-Gaussian system. In this paper, we combine a canonical variable analysis (CVA) and adaptive kernel density estimation (AKDE) for the early fault detection of nonlinear dynamic hydraulic systems. The collected hydraulic system data set was used to construct the typical variable space, and the state space and residual space are divided to represent the characteristics of different correlations between the two variables, which are quantitatively described using Hotelling's <i>T</i><sup>2</sup> and <i>Q</i>. In order to investigate the proper upper control limits, AKDE was utilised to estimate the underlying probability density functions of <i>T</i><sup>2</sup> and <i>Q</i> by taking the nonlinearity of the hydraulic system variables into consideration. The advantages of the proposed approach for incipient fault detection are illustrated via a marine power plant lubrication system.
Cited by 3Open access
M. G. S. Vaz, Carmen L. T. Borges · 2026
This paper explores a method for preserving correlations between random variables in the state-space sampling technique that is adopted in Non-Sequential Monte Carlo Simulation. The proposed method combines the Kernel Density Estimation technique with Bayesian Networks to encode dependency relationships between variables. The model involves transforming historical input data through normalization and aggregation. The algorithm was evaluated using actual renewable generation (wind and solar) and load time series for the Northeast region of Brazil. The original and sampled series were then compared using scatter plots and correlation metrics that validated the effectiveness of the explored method.
Cited by 0
Kota Matsui, Wataru Kumagai, Kenta Kanamori et al. · arXiv (Cornell University) · 2018
In this paper, we propose a variable selection method for general nonparametric kernel-based estimation. The proposed method consists of two-stage estimation: (1) construct a consistent estimator of the target function, (2) approximate the estimator using a few variables by l1-type penalized estimation. We see that the proposed method can be applied to various kernel nonparametric estimation such as kernel ridge regression, kernel-based density and density-ratio estimation. We prove that the proposed method has the property of the variable selection consistency when the power series kernel is used. This result is regarded as an extension of the variable selection consistency for the non-negative garrote to the kernel-based estimators. Several experiments including simulation studies and real data applications show the effectiveness of the proposed method.
Cited by 0Open access
Dragoljub Pokrajac, Longin Jan Latecki, Aleksandar Lazarević et al. · 2007
Estimation of probability density functions based on available data is important problem arising in various fields, such as telecommunications, machine learning, data mining, pattern recognition and computer vision. In this paper, we consider Kernel-based non-parametric density estimation methods and derive formulae for variable kernel density estimation using generalized, elliptic Gaussian kernels. The proposed technique is verified on simulated data.
Cited by 5
Muhammad Aslam · New Mathematics and Natural Computation · 2026
Kernel density estimation is a popular non-parametric method for estimating the probability density function of a random variable. However, classical kernel density estimation assumes all observations are exact, limiting its applicability in scenarios involving uncertainty or indeterminacy. To overcome this limitation, we introduce a generalized interval-valued method based kernel density estimation framework based on generalized interval-valued method. This method integrates indeterminacy into the estimation process, enhancing flexibility and robustness. Using Gaussian and Epanechnikov kernels, we extend classical bandwidth selection theory to incorporate generalized interval-valued method parameters. Comprehensive studies, including train and test models, highlight its applications in artificial intelligence and environmental data analysis. The results demonstrate that generalized interval-valued method based kernel density estimation effectively handles uncertainty, improving accuracy in estimating probability density functions under imprecise data conditions. This advancement significantly broadens the utility of kernel density estimation, making it highly suitable for complex and uncertain datasets in various domains.
Cited by 0
А. V. Lapko, В. А. Лапко · Optoelectronics Instrumentation and Data Processing · 2024
The methods of selecting the bandwidth of kernel functions for the regression estimation of the probability density of a one-dimensional random variable are studied. The regression estimation of the probability density is a modification of Rosenblatt–Parzen statistics and used in big statistical data processing. Its synthesis is based on the compression of an initial sample by decomposing the range of random variable values. Resulting data array elements are the centers of sampling intervals and the frequencies of belonging to them for random variable values from the initial sample. This information is sufficient to estimate the probability density of a random variable in the form of nonparametric regression. Therefore, it becomes possible to select the bandwidth for the kernel functions of regression estimation from the condition of its minimum approximation error for the desired probability density. The traditional approach of nonparametric estimation optimization to the probability density is based on the minimization of its mean-square deviation. The approximation properties of regression estimation for the probability density are analyzed by using the considered methods of its optimization.
Cited by 0
Aleksandra Baszczyńska · The Central European Review of Economics and Management · 2016
For a random variable with bounded support, the kernel estimation of functional characteristics may lead to the occurrence of the so-called boundary effect. In the case of the kernel density estimation it can mean an increase of the estimator bias in the areas near the ends of the support, and can lead to a situation where the estimator is not a density function in the support of a random variable. In the paper the procedures for reducing boundary effect for kernel estimators of density function, distribution function and regression function are analyzed. Modifications of the classical kernel estimators and examples of applications of these procedures in the analysis of the functional characteristics relating to gross national product per capita are presented. The advantages of procedures are indicated taking into account the reduction of the bias in the boundary region of the support of the random variable considered.
Cited by 1Open access
А. V. Lapko, В. А. Лапко · Optoelectronics Instrumentation and Data Processing · 2023
A method for fast selection of bandwidths of kernel functions in the nonparametric estimation of a two-dimensional random variable with dependent components is proposed. The method is based on results of the analysis of asymptotic properties of the Rosenblatt–Parzen kernel probability density estimate. The properties of a fast algorithm for bandwidth selection in the considered nonparametric estimation of the probability density are investigated.
Cited by 0