Search 250 million research papers

Multiple Poisson kernels

Spreafico M · BOA (University of Milano-Bicocca) · 2009

Cited by 0

Active Learning With Multiple Kernels

Song‐Nam Hong, Jeongmin Chae · IEEE Transactions on Neural Networks and Learning Systems · 2021

Online multiple kernel learning (OMKL) has provided an attractive performance in nonlinear function learning tasks. Leveraging a random feature (RF) approximation, the major drawback of OMKL, known as the curse of dimensionality, has been recently alleviated. These advantages enable RF-based OMKL to be considered in practice. In this article, we introduce a new research problem, named stream-based active MKL (AMKL), in which a learner is allowed to label some selected data from an oracle according to a selection criterion. This is necessary for many real-world applications as acquiring a true label is costly or time consuming. We theoretically prove that the proposed AMKL achieves an optimal sublinear regret O(√T) as in OMKL with little labeled data, implying that the proposed selection criterion indeed avoids unnecessary label requests. Furthermore, we present AMKL with an adaptive kernel selection (named AMKL-AKS) in which irrelevant kernels can be excluded from a kernel dictionary "on the fly." This approach improves the efficiency of active learning and the accuracy of function learning. Via numerical tests with real data sets, we verify the superiority of AMKL-AKS, yielding a similar accuracy performance with OMKL counterpart using a fewer number of labeled data.

Cited by 20

Multiple kernels optimization in GPU

Yu Xin Sun · DR-NTU (Nanyang Technological University) · 2012

This project is developed in the NVIDIA CUDA C/C++ environment which is provided. All the equipment and software stacks are provided by Parallel and Distributed Computing Center. The objective of this project is to find a way to split the GPU kernel and perform GPU kernel scheduling based on the importance of GPU kernels. \n \n \n \nGeneral purpose computing GPU (GPGPU) is burgeoning technique to enhance the computation of parallel programs. However, the non-preemptive nature of GPU kernel possesses great challenges for applying GPU computing to real-time application. In this project, we proposed a solution to split GPU kernels from the host code and also developed a model to schedule the GPU kernels with the kernel splitting techniques proposed. \n \n \n \nIn the first part of this project, we focus on how to split GPU kernels and make GPU kernels preemptive. Our design is to split GPU kernel into smaller size of sub kernels by reducing the grid of blocks of threads to execute it. Then this kernel is executed multiple times to have same functionality as the original kernel. In order to this, kernel invocation is introduced additional parameters by modifying the PTX code. \n \n \n \nThe second part of this project is to develop a model that can perform GPU kernel scheduling based on the priority assigned with the kernel splitting techniques developed in the first part. Our design is a server-clients model. In this model, clients initiate the kernel invocation, and pass control to server, then the server handle the actual invocation of GPU kernels. Server takes the overcharge the execution of all the kernels so it can schedule them based on priority.

Cited by 0

Fuzzy clustering with Multiple Kernels

Naouel Baili, Hichem Frigui · 2011

In this paper, the kernel fuzzy c-means clustering algorithm is extended to an adaptive cluster model which maps data points to a high dimensional feature space through an optimal convex combination of homogenous kernels with respect to each cluster. This generalized model, called Fuzzy C Means with Multiple Kernels (FCM-MK), strives to find a good partitioning of the data into meaningful clusters and the optimal kernel-induced feature map in a completely unsupervised way. It constructs the kernel from a number of Gaussian kernels and learns a resolution specific weight for each kernel function in each cluster. This allows better characterization and adaptability to each individual cluster. The effectiveness of the proposed algorithm is demonstrated for several toy and real data sets.

Cited by 15

Derivatives, Differences, Multiple Fourier Kernels.

Detian Liu, Louise Young · Defense Technical Information Center (DTIC) · 1976

Identities and inequalities for Fourier kernels and for difference operators are related to a geometric series identity. The resulting machinery is applied to obtain, in the approximation theory for ordinary or partial derivatives of any order, necessary and sufficient conditions in place of classical sufficient conditions. Alternative formulations are given in terms of Tauberian Theorems, and in terms of Schwartz distributions. The results are achieved by making use, as in L. C. Young's papers on Stochastic integrals and the like, of pairs of estimate functions in place of the classical higher moduli of continuity.

Cited by 0

Incremental fuzzy clustering with multiple kernels

Naouel Baili, Hichem Frigui · 2014

This paper presents two incremental clustering algorithms based on FCMK, a fuzzy clustering with multiple kernels algorithm we developed earlier [1]. The FCMK algorithm has a memory requirement of O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), where N is the number of objects in the data set. Thus, even data sets that have nearly 1, 000, 000 objects require terabytes of working memory-impractical for most computers. One way to attack this problem is by using incremental algorithms; these algorithms sequentially process chunks or samples of the data, combining the results from each chunk. The proposed incremental algorithms neither use any complicated data structure nor any complicated data compression techniques, yet produce data partitions comparable to FCMK. We assess the performance of our incremental algorithms by, first, comparing their clustering results to that of the FCMK and, second, by showing that these algorithms can produce reasonable partitions of large data sets.

Cited by 2

Multiple-guard kernels of simple polygons

Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 1999

Cited by 0

Learning local graph from multiple kernels

Zheng Liu, Shiluo Huang, Wei Jin et al. · Neurocomputing · 2023

Cited by 4

Non-Sparse Regularization with Multiple Kernels

Kloft, Marius, Author, Brefeld, Ulf, Author, Sonnenburg, Soeren, Author et al. · UC Berkeley

Learning linear combinations of multiple kernels is an appealing strategy when the right choice of features is unknown. Previous approaches to multiple kernel learning (MKL) promote sparse kernel combinations to support interpretability and scalability. Unfortunately, this lp-norm MKL is rarely observed to outperform trivial baselines in practical applications. To allow for robust kernel mixtures, we generalize MKL to arbitrary norms. We devise new insights on the connection between several existing MKL formulations and develop two efficient interleaved optimization strategies for arbitrary norms, like lp-norms with p&gt;1. Empirically, we demonstrate that the interleaved optimization strategies are much faster compared to the commonly used wrapper approaches. An experiment on controlled artificial data experiment sheds light on the appropriateness of sparse, non-sparse and uniform MKL in various scenarios. Application of lp-norm MKL to three hard real-world problems from computational biology show that non-sparse MKL achieves accuracies that go beyond the state-of-the-art. We conclude that our improvements finally made MKL fit for deployment to practical applications: MKL now has a good chance of improving the accuracy (over a plain sum kernel) at an affordable computational cost.

Cited by 0

Kernels for Generalized Multiple-Instance Learning

Qingping Tao, Stephen Scott, N. V. Vinodchandran et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2008

The multiple-instance learning (MIL) model has been successful in numerous application areas. Recently, a generalization of this model and an algorithm for it were introduced, showing significant advantages over the conventional MIL model on certain application areas. Unfortunately, that algorithm is not scalable to high dimensions. We adapt that algorithm to one using a support vector machine with our new kernel k\wedge. This reduces the time complexity from exponential in the dimension to polynomial. Computing our new kernel is equivalent to counting the number of boxes in a discrete, bounded space that contain at least one point from each of two multisets. We show that this problem is #P-complete, but then give a fully polynomial randomized approximation scheme (FPRAS) for it. We then extend k\wedge by enriching its representation into a new kernel kmin, and also consider a normalized version of k\wedge that we call k\wedge/\vee (which may or may not not be a kernel, but whose approximation yielded positive semidefinite Gram matrices in practice). We then empirically evaluate all three measures on data from content-based image retrieval, biological sequence analysis, and the musk data sets. We found that our kernels performed well on all data sets relative to algorithms in the conventional MIL model.

Cited by 17

Distributed Online Learning With Multiple Kernels

Song‐Nam Hong, Jeongmin Chae · IEEE Transactions on Neural Networks and Learning Systems · 2021

We consider the problem of learning a nonlinear function over a network of learners in a fully decentralized fashion. Online learning is additionally assumed, where every learner receives continuous streaming data locally. This learning model is called a fully distributed online learning (or a fully decentralized online federated learning). For this model, we propose a novel learning framework with multiple kernels, which is named DOMKL. The proposed DOMKL is devised by harnessing the principles of an online alternating direction method of multipliers and a distributed Hedge algorithm. We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret O(√T) , implying that every learner in the network can learn a common function having a diminishing gap from the best function in hindsight. Our analysis also reveals that DOMKL yields the same asymptotic performance as the state-of-the-art centralized approach while keeping local data at edge learners. Via numerical tests with real datasets, we demonstrate the effectiveness of the proposed DOMKL on various online regression and time-series prediction tasks.

Cited by 32

Multiple Kernel Learning with Many Kernels

Arash Afkanpour · University of Alberta Library · 2013

Multiple kernel learning (MKL) addresses the problem of learning the kernel function from data. Since a kernel function is associated with an underlying feature space, MKL can be considered as a systematic approach to feature selection. Many of the existing MKL algorithms perform kernel learning by combining a given set of base kernels. While efficient MKL algorithms are capable of handling large training sets, their computational complexity depends linearly on the number of base kernels. Hence, these algorithms are not scalable to problems with many kernels. In this thesis, we investigate MKL when the number of kernels is very large. We aim to exploit the structure of kernel space to design efficient MKL algorithms. Such a structure exists for some of the important families of kernels, such as polynomial kernels and Gaussian kernels. We propose efficient gradient-based algorithms, with convergence guarantees, for the two prominent problem formulations of MKL, i.e. one-stage and two-stage MKL. We show that stochastic gradient descent and greedy coordinate descent are suitable algorithms to deal with very large kernel sets. We propose an efficient stochastic gradient descent method for one-stage MKL. We show that sampling a coordinate with a probability proportional to the magnitude of gradient results in a low-variance estimate of gradient. For the case of learning polynomial kernels we show that the computational complexity of our algorithm has only a logarithmic dependence on the number of kernels. We show how greedy coordinate descent can be applied to one-stage and two-stage MKL. Greedy coordinate descent is in particular useful when the goal is to combine continuously-parameterized kernels, such as Gaussian kernels. We perform thorough empirical study on synthetic and real data to compare the new gradient-based algorithms against several MKL algorithms. Our experiments demonstrate that our new methods are competitive in terms of generalization performance, while their computational cost is significantly less than other methods that enjoy similarly good generalization performance.

Cited by 0Open access

Multiple Kernel k-Means with Incomplete Kernels

Xinwang Liu, Miaomiao Li, Lei Wang et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 2017

Multiple kernel clustering (MKC) algorithms optimally combine a group of pre-specified base kernels to improve clustering performance. However, existing MKC algorithms cannot efficiently address the situation where some rows and columns of base kernels are absent. This paper proposes a simple while effective algorithm to address this issue. Different from existing approaches where incomplete kernels are firstly imputed and a standard MKC algorithm is applied to the imputed kernels, our algorithm integrates imputation and clustering into a unified learning procedure. Specifically, we perform multiple kernel clustering directly with the presence of incomplete kernels, which are treated as auxiliary variables to be jointly optimized. Our algorithm does not require that there be at least one complete base kernel over all the samples. Also, it adaptively imputes incomplete kernels and combines them to best serve clustering. A three-step iterative algorithm with proved convergence is designed to solve the resultant optimization problem. Extensive experiments are conducted on four benchmark data sets to compare the proposed algorithm with existing imputation-based methods. Our algorithm consistently achieves superior performance and the improvement becomes more significant with increasing missing ratio, verifying the effectiveness and advantages of the proposed joint imputation and clustering.

Cited by 75Open access

Multiple Kernel Learning with High Order Kernels

Shuhui Wang, Shuqiang Jiang, Qingming Huang et al. · 2010

Previous Multiple Kernel Learning approaches (MKL) employ different kernels by their linear combination. Though some improvements have been achieved over methods using single kernel, the advantages of employing multiple kernels for machine learning are far from being fully developed. In this paper, we propose to use “high order kernels” to enhance the learning of MKL when a set of original kernels are given. High order kernels are generated by the products of real power of the original kernels. We incorporate the original kernels and high order kernels into a unified localized kernel logistic regression model. To avoid over-fitting, we apply group LASSO regularization to the kernel coefficients of each training sample. Experiments on image classification prove that our approach outperforms many of the existing MKL approaches.

Cited by 9

B.: Kernels for generalized multiple-instance learning

Qingping Tao, Stephen D. Scott, N. V. Vinodch et al. · 2008

Abstract—The multiple-instance learning (MIL) model has been successful in numerous application areas. Recently, a generalization of this model and an algorithm for it have been introduced, showing significant advantages over the conventional MIL model on certain application areas. Unfortunately, that algorithm is not scalable to high dimensions. We adapt that algorithm to one that uses a support vector machine with our new kernel k^. This reduces the time complexity from exponential in the dimension to polynomial. Computing our new kernel is equivalent to counting the number of boxes in a discrete bounded space that contain at least one point from each of two multisets. We first show that this problem is #P-complete and then present a fully polynomial randomized approximation scheme (FPRAS) for it. We then extend k ^ by enriching its representation into a new kernel kmin and also consider a normalized version of k^ that we call k ^= _ (which may or may not be a kernel but whose approximation yielded positive semidefinite Gram matrices in practice). We then empirically evaluate all three measures on data from content-based image retrieval, biological sequence analysis, and the Musk data sets. We found that our kernels performed well on all data sets relative to algorithms in the conventional MIL model. Index Terms—Kernels, support vector machines, generalized multiple-instance learning, content-based image retrieval, biological sequence analysis, fully polynomial randomized approximation schemes. Ç 1

Cited by 0

Learning with multiple kernels : algorithms and applications

Xiao Hao · 2014

Multiple kernel learning (MKL) is a family of machine learning techniques by learning with multiple kernel functions/matrices. It has been demonstrated as a promising technique for solving many real-world applications, especially for its power of exploiting multiple kernels for fusing diverse information from multiple sources. Beyond the traditional MKL studies for classification, this thesis investigates a more general research problem of learning with multiple kernels for different applications. Specifically, thesis has proposed a family of novel algorithms for learning with multiple kernels to tackle classification, hashing and similarity learning tasks, respectively. The following gives the main contributions of the proposed techniques. Firstly, we aim to investigate novel learning with multiple kernels techniques for classification tasks. Specifically, this thesis presents a novel framework of Multiple Kernel Boosting (MKBoost) for classification tasks, which applies the idea of boosting techniques to learn kernel-based classifiers with multiple kernels for resolving classification problems. Conventional MKL methods often formulate the problem as an optimization task of learning the optimal combinations of both kernels and classifiers, which usually results in some forms of challenging optimization tasks that are often difficult to be solved. Different from the existing MKL methods, in this thesis, we investigate a boosting framework of multiple kernel learning for classification tasks, i.e., we adopt boosting to solve a variant of MKL problem, which avoids solving the complicated optimization tasks. Based on the proposed framework, we propose several variants of MKBoost algorithms and extensively examine their empirical performance on a number of benchmark datasets in comparisons to various state-of-the-art MKL algorithms on classification tasks. Experimental results show that the proposed method is more effective and efficient than the existing MKL techniques. Secondly, we aim to investigate novel learning with multiple kernels techniques for hashing tasks in multimedia similarity search. Specifically, this thesis proposes a Boosting Multi-Kernel Locality-Sensitive Hashing (BMKLSH) scheme that significantly boosts the retrieval performance of existing hashing methods by making use of multiple kernels. Similarity search is a key challenge for multimedia retrieval applications where data are usually represented in high-dimensional space. Among various algorithms proposed for similarity search in high-dimensional space, Locality-Sensitive Hashing (LSH) is the most popular one, which recently has been extended to Kernelized Locality-Sensitive Hashing (KLSH) by exploiting kernel similarity for better retrieval efficacy. Typically, KLSH works only with a single kernel, which is often limited in real-world multimedia applications, where data may originate from multiple resources or can be represented in several different forms. For example, in content-based multimedia retrieval, a variety of features can be extracted to represent contents of an image. So BMKLSH is proposed to overcome the limitation of regular KLSH. We conduct extensive experiments for large-scale content-based image retrieval, in which encouraging results show that the proposed method outperforms the state-of-the-art techniques. Finally, we aim to investigate novel learning with multiple kernels techniques for learning similarity functions in multimedia retrieval. This thesis proposes a novel Online Multiple Kernel Similarity (OMKS) learning method, which learns a flexible nonlinear proximity function with multiple kernels to improve visual similarity search in CBIR. Recent years have witnessed a number of studies on distance metric learning to improve visual similarity search in CBIR. Despite their successes, most existing methods on distance metric learning are limited in two aspects. First, they usually assume the target proximity function follows the family of Mahalanobis distances, which limits their capacity of measuring similarity of complex patterns in real applications. Second, they often cannot effectively handle the similarity measure of multi-modal data that may originate from multiple resources. To overcome these limitations, this thesis investigates an online kernel similarity learning framework for learning kernel-based proximity functions, which goes beyond the conventional linear distance metric learning approaches. Based on the framework, OMKS is proposed. We evaluate the proposed technique for CBIR on a variety of image data sets, in which encouraging results show that OMKS outperforms the state-of-the-art techniques significantly.

Cited by 0Open access

Multiple Kernel k-Means with Incomplete Kernels

Gao, Wen, Liu, Xinwang, Yin, Jianping et al. · Research Online (University of Wollongong) · 2026

Multiple kernel clustering (MKC) algorithms optimally combine a group of pre-specified base kernel matrices to improve clustering performance. However, existing MKC algorithms cannot efficiently address the situation where some rows and columns of base kernel matrices are absent. This paper proposes two simple yet effective algorithms to address this issue. Different from existing approaches where incomplete kernel matrices are first imputed and a standard MKC algorithm is applied to the imputed kernel matrices, our first algorithm integrates imputation and clustering into a unified learning procedure. Specifically, we perform multiple kernel clustering directly with the presence of incomplete kernel matrices, which are treated as auxiliary variables to be jointly optimized. Our algorithm does not require that there be at least one complete base kernel matrix over all the samples. Also, it adaptively imputes incomplete kernel matrices and combines them to best serve clustering. Moreover, we further improve this algorithm by encouraging these incomplete kernel matrices to mutually complete each other. The three-step iterative algorithm is designed to solve the resultant optimization problems. After that, we theoretically study the generalization bound of the proposed algorithms. Extensive experiments are conducted on 13 benchmark data sets to compare the proposed algorithms with existing imputation-based methods. Our algorithms consistently achieve superior performance and the improvement becomes more significant with increasing missing ratio, verifying the effectiveness and advantages of the proposed joint imputation and clustering.

Cited by 0Open access

Personalized Online Federated Learning with Multiple Kernels

Pouya M. Ghari, Yanning Shen · arXiv (Cornell University) · 2023

Multi-kernel learning (MKL) exhibits well-documented performance in online non-linear function approximation. Federated learning enables a group of learners (called clients) to train an MKL model on the data distributed among clients to perform online non-linear function approximation. There are some challenges in online federated MKL that need to be addressed: i) Communication efficiency especially when a large number of kernels are considered ii) Heterogeneous data distribution among clients. The present paper develops an algorithmic framework to enable clients to communicate with the server to send their updates with affordable communication cost while clients employ a large dictionary of kernels. Utilizing random feature (RF) approximation, the present paper proposes scalable online federated MKL algorithm. We prove that using the proposed online federated MKL algorithm, each client enjoys sub-linear regret with respect to the RF approximation of its best kernel in hindsight, which indicates that the proposed algorithm can effectively deal with heterogeneity of the data distributed among clients. Experimental results on real datasets showcase the advantages of the proposed algorithm compared with other online federated kernel learning ones.

Cited by 2Open access

Optimal Neighborhood Kernel Clustering with Multiple Kernels

Xinwang Liu, Sihang Zhou, Yueqing Wang et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 2017

Multiple kernel $k$-means (MKKM) aims to improve clustering performance by learning an optimal kernel, which is usually assumed to be a linear combination of a group of pre-specified base kernels. However, we observe that this assumption could: i) cause limited kernel representation capability; and ii) not sufficiently consider the negotiation between the process of learning the optimal kernel and that of clustering, leading to unsatisfying clustering performance. To address these issues, we propose an optimal neighborhood kernel clustering (ONKC) algorithm to enhance the representability of the optimal kernel and strengthen the negotiation between kernel learning and clustering. We theoretically justify this ONKC by revealing its connection with existing MKKM algorithms. Furthermore, this justification shows that existing MKKM algorithms can be viewed as a special case of our approach and indicates the extendability of the proposed ONKC for designing better clustering algorithms. An efficient algorithm with proved convergence is designed to solve the resultant optimization problem. Extensive experiments have been conducted to evaluate the clustering performance of the proposed algorithm. As demonstrated, our algorithm significantly outperforms the state-of-the-art ones in the literature, verifying the effectiveness and advantages of ONKC.

Cited by 118Open access

Multiple kernels matching for precise television guidance

Shiyin Qin · Optical Engineering · 2011

This research analyzed key technologies, including object tracking and image matching, for realizing a television guidance system and proposed multiple kernels matching for precise television guidance. A semiphysical simulation system was designed to test the effectiveness of the proposed method. The experimental results showed that this method performed well in a television guidance system and did better than a template matching method.

Cited by 0