Tetsuji Kuboyama, Kouichi Hirata, Hisashi Kashima et al. · Transactions of the Japanese Society for Artificial Intelligence · 2007
Learning from tree-structured data has received increasing interest with the rapid growth of tree-encodable data in the World Wide Web, in biology, and in other areas. Our kernel function measures the similarity between two trees by counting the number of shared sub-patterns called tree q-grams, and runs, in effect, in linear time with respect to the number of tree nodes. We apply our kernel function with a support vector machine (SVM) to classify biological data, the glycans of several blood components. The experimental results show that our kernel function performs as well as one exclusively tailored to glycan properties.
Cited by 38Open access
Souad Taouti, Hadda Cherroun, Djelloul Ziadi · arXiv (Cornell University) · 2023
Tree kernels are fundamental tools that have been leveraged in many applications, particularly those based on machine learning for Natural Language Processing tasks. In this paper, we devise a parallel implementation of the sequential algorithm for the computation of some tree kernels of two finite sets of trees (Ouali-Sebti, 2015). Our comparison is narrowed on a sequential implementation of SubTree kernel computation. This latter is mainly reduced to an intersection of weighted tree automata. Our approach relies on the nature of the data parallelism source inherent in this computation by deploying the MapReduce paradigm. One of the key benefits of our approach is its versatility in being adaptable to a wide range of substructure tree kernel-based learning methods. To evaluate the efficacy of our parallel approach, we conducted a series of experiments that compared it against the sequential version using a diverse set of synthetic tree language datasets that were manually crafted for our analysis. The reached results clearly demonstrate that the proposed parallel algorithm outperforms the sequential one in terms of latency.
Cited by 0Open access
Souad Taouti, Hadda Cherroun, Djelloul Ziadi · 2024
International audience
Cited by 0Open access
Tyler J. Gray (6162674), Christopher M. Danforth (8345841), Peter Sheridan Dodds (8900321) · Figshare · 2020
<p>The root node represents ‘h’. From there, branching to the left (light gray edge) is equivalent to appending an ‘h’. Branching to the right (dark gray edge) is equivalent to appending an ‘a’. The edge width is logarithmically related to the number of tokens that pass along that edge when spelled out. A few example words are annotated, and their corresponding nodes are denoted with a star. This tree was trimmed by only including words with a token count of at least 10,000. The code used to create the figures for these spelling trees is largely based on the algorithm presented by Wetherel and Shannon [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0232938#pone.0232938.ref038" target="_blank">38</a>]. We note that Mill has written a more recent paper based largely on this earlier work specialized for Python [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0232938#pone.0232938.ref039" target="_blank">39</a>], and an implementation for it as well [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0232938#pone.0232938.ref040" target="_blank">40</a>], but they both contain algorithmic bugs (detailed in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0232938#pone.0232938.s003" target="_blank">S3 Appendix</a>).</p>
Cited by 0Open access
Giovanni Da San Martino, Nicolò Navarin, Alessandro Sperduti · 2012
This paper proposes a new tree-based kernel for graphs. Graphs are decomposed into multisets of ordered Directed Acyclic Graphs (DAGs) and a family of kernels computed by application of tree kernels extended to the DAG domain. We focus our attention on the efficient development of one member of this family. A technique for speeding up the computation is given, as well as theoretical bounds and practical evidence of its feasibility. State of the art results on various benchmark datasets prove the effectiveness of our approach.
Cited by 46
Moschitti, Alessandro, Pighin, Daniele · Institutional Research Information System (Università degli Studi di Trento) · 2006
Cited by 2
SUN JUN, Zhang, M., Tan, C.L. · National University of Singapore · 2011
Proceedings of the National Conference on Artificial Intelligence
Cited by 0
Jun Sun, Min Zhang, Chew Lim Tan · Proceedings of the AAAI Conference on Artificial Intelligence · 2011
We propose Tree Sequence Kernel (TSK), which implicitly exhausts the structure features of a sequence of subtrees embedded in the phrasal parse tree. By incorporating the capability of sequence kernel, TSK enriches tree kernel with tree sequence features so that it may provide additional useful patterns for machine learning applications. Two approaches of penalizing the substructures are proposed and both can be accomplished by efficient algorithms via dynamic programming. Evaluations are performed on two natural language tasks, i.e. Question Classification and Relation Extraction. Experimental results suggest that TSK outperforms tree kernel for both tasks, which also reveals that the structure features made up of multiple subtrees are effective and play a complementary role to the single tree structure.
Cited by 18Open access
Marco Cuturi, Jean‐Philippe Vert · Neural Networks · 2005
Cited by 41
Marwah Alian · Procedia Computer Science · 2024
The aim of paraphrasing identification techniques is to identify if two sentences or texts have the same meaning; even if they do not contain a number of identical phrases or words. In this paper we proposed a hybrid technique based on using attention constituency vector (ACV)-tree kernel in short texts or sentence similarity computation. Then a similarity threshold is used to identifying if these sentences are paraphrased or not. The experiments are conducted on Arabic paraphrasing benchmark. The proposed method provides a recall of 70% and a precision of 76%, when the determined threshold is 0.5, while a recall of 94% and a precision of 0.751 achieved using a threshold of 0.3.
Cited by 1Open access
Jiang Qiang-rong, Li hualan, Gao Yuan · 2010
Pattern recognition algorithms are facing the challenge to deal with an increasing number of complex objects. For graph data, a whole toolbox of pattern recognition algorithms becomes available by defining a kernel function on instances of graphs. Graph similarity is the central problem for all learning tasks such as clustering and classification on graphs. Graph kernels based on walks, shortest path, sub-trees and cycles in graphs have been proposed so far. As a general problem, these kernels are either computationally expensive or limited in their expressiveness. We try to overcome this problem by defining expressive graph kernels which are based on cycle of undirected graph. Cycle kernels of undirected graph is computable in polynomial time, retain expressivity and are still positive definite. In experiments on classification of graph models of face images, our cycle kernels based on spanning tree of undirected graph show significantly higher classification accuracy than walk kernel.
Cited by 7
Dmitry Devyatkin, O. G. Grigoriev · Scientific and Technical Information Processing · 2023
Axis-parallel decision trees perform poorly on that multidimensional sparse data that are frequently input in many tasks. A straightforward solution is to create decision trees that have oblique splits; however, most training approaches have low performance. These models can easily overfit, so they should be combined with a random ensemble. This paper proposes an algorithm to train kernel decision trees. At each stump, the algorithm optimizes a loss function with a margin rescaling approach that simultaneously optimizes the margin and impurity criteria. We performed an experimental evaluation of several tasks, such as studying the reaction of social media users and image recognition. The experimental results show that the proposed algorithm trains ensembles that outperform other oblique or kernel forests in many datasets.
Cited by 0
Guodong Zhou · 2011
This paper introduced a learning approach of Tree Kernel-based Zero Pronouns Recognition,and presented a system based-upon this approach.In order to get the syntax structure tree,which includes the zero pronoun,it marks the sentences of the CTB manually,and constructs a base-corpus.Then,it proposed three different policies to get syntax structure tree,which can be trained by SVM as a feature.At last,it also presented a baseline system based on rules,with which we can compares.Experimental results show that the recognition rate of the approach based on tree kernel is better than the one of the System based on rules.
Cited by 0
Wei Zhang, Peifeng Li, Qiaoming Zhu · 2010
Sentiment classification is a way to analyze the subjective information in the text and then mine the opinion. We focus on the sentence-level sentiment classification. On the systematically analyzing the importance and difficulties of the sentence-level sentiment classification, this paper proposes a syntax tree pruning and tree kernel-based approach to sentiment classification. In our method, the convolution kernel of SVM is first used to obtain structured information, and then apply syntax tree as a feature in Sentiment Classification. Firstly, we focus on how to apply the structured features from the syntax tree to the sentiment classification and propose a novel approach of sentence-level sentiment classification which apply the tree kernel and composite kernel to the SVM classifier. Secondly, we provide two kinds of syntax tree pruning strategies: adjectives-based and sentiment words-based. The experimental results show that our method can achieve better performance in sentence level Sentiment Classification.
Cited by 8
Palmirani, Monica · Open Repository and Bibliography (University of Luxembourg) · 2022
Cited by 0Open access
Ryan R. Curtin, Parikshit Ram · Statistical Analysis and Data Mining The ASA Data Science Journal · 2014
Abstract The problem of max‐kernel search arises everywhere: given a query point \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$p_q$ \end{document} , a set of reference objects \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$S_r$ \end{document} and some kernel \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$\mathcal{K}$ \end{document} , find \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$arg\,max_{p_r \in S_r} \mathcal{K}(p_q, p_r)$ \end{document} . Max‐kernel search is ubiquitous and appears in countless domains of science, thanks to the wide applicability of kernels. A few domains include image matching, information retrieval, bio‐informatics, similarity search, and collaborative filtering (to name just a few). However, there is no generalized technique for efficiently solving max‐kernel search. This paper presents a single‐tree algorithm called single‐tree FastMKS which returns the max‐kernel solution for a single query point in provably \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$O(\log N)$ \end{document} time (where \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$N$ \end{document} is the number of reference objects), and also a dual‐tree algorithm ( dual‐tree FastMKS ) which is useful for max‐kernel search with many query points. If the set of query points is of size \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$O(N)$ \end{document} , this algorithm returns a solution in provably \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$O(N)$ \end{document} time, which is significantly better than the \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$O(N^2)$ \end{document} linear scan solution; these bounds are dependent on the expansion constant of the data. These algorithms work for abstract objects, as they do not require explicit representation of the points in kernel space. Empirical results for a variety of datasets show up to five orders of magnitude speedup in some cases. In addition, we present approximate extensions of the FastMKS algorithms that can achieve further speedups.
Cited by 25
Le Song, Eric P. Xing, Ankur P. Parikh · Figshare · 2018
Latent tree graphical models are natural tools for expressing long range and hierarchical dependencies among many variables which are common in computer vision, bioinformatics and natural language processing problems. However, existing models are largely restricted to discrete and Gaussian variables due to computational constraints; furthermore, algorithms for estimating the latent tree structure and learning the model parameters are largely restricted to heuristic local search. We present a method based on kernel embeddings of distributions for latent tree graphical models with continuous and non-Gaussian variables. Our method can recover the latent tree structures with provable guarantees and perform local-minimum free parameter learning and efficient inference. Experiments on simulated and real data show the advantage of our proposed approach.
Cited by 30Open access
Daniele Pighin · Unitn-eprints PhD (University of Trento) · 2010
Tree Kernel functions are powerful tools for solving different classes of problems requiring large amounts of structured information. Combined with accurate learning algorithms, such as Support Vector Machines, they allow us to directly encode rich syntactic data in our learning problems without requiring an explicit feature mapping function or deep specific domain knowledge. \n \nHowever, as other very high dimensional kernel families, they come with two major drawbacks: first, the computational complexity induced by the dual representation makes them unpractical for very large datasets or for situations where very fast classifiers are necessary, e.g. real time systems or web applications; second, their implicit nature somehow limits their scientific appeal, as the implicit models that we learn cannot cast new light on the studied problems. \n \nAs a possible solution to these two problems, this Thesis presents an approach to feature selection for tree kernel functions in the context of Support Vector learning, based on a greedy exploration of the fragment space. Features are selected according to a gradient norm preservation criterion, i.e. we select the heaviest features that account for a large percentage of the gradient norm, and are explicitly modeled and represented. The result of the feature extraction process is a data structure that can be used to decode the input structured data, i.e. to explicitly describe a tree in terms of its more relevant fragments. \n \nWe present theoretical insights that justify the adopted strategy and detail the algorithms and data structures used to explore the feature space and store the most relevant features. Experiments on three different multi-class NLP tasks and data sets, namely question classification, relation extraction and semantic role labeling, confirm the theoretical findings and show that the decoding process can produce very fast and accurate linear classifiers, along with the explicit representation of the most relevant structured features identified for each class.
Cited by 0Open access
Ro Moschitti, Daniele Pighin, Roberto Basili · 2008
Abstract. Recent work on the design of automatic systems for semantic role labeling has shown that such task is complex from both modeling and implementation point of views. Tree kernels alleviate such complexity as kernel functions generate features automatically and require less software development for data pre-processing. In this paper, we study several tree kernel approaches for boundary detection, argument classification and, most notably, proposition re-ranking. The comparative experiments on Support Vector Machines with such kernels on the CoNLL 2005 dataset show that very simple tree manipulations trigger automatic feature engineering that highly improves accuracy and efficiency in every SRL phase. 1
Cited by 0
Art F. Y. Poon (125509), Lorne W. Walker (478847), Heather Murray (22128) et al. · Figshare · 2015
<p>For trees comprising and nodes, respectively, there are pairs of nodes to evaluate. (A) Starting from a given pair of nodes (indicated in figure by circles with double-outlines), the algorithm finds the largest common subset tree rooted at these nodes. First, we find that for both nodes, neither of the branches terminate at a ‘leaf node’ (marked with ‘‘). This match contributes a relatively small amount to our kernel score, not only because the matching subset trees (highlighted in thick blue lines) comprise only one node each, but also because their discordant branch lengths lead to a substantial penalty. (B) Next, we descend down the left branch in both trees. The current nodes (open circles) in both trees spawn one leaf node and one internal node; therefore, the subset trees continue to match. In addition, their branch lengths are similar, so their contribution to the cumulative kernel score is given greater weight. (C) Finally, we descend down the right branch in both trees and find that the subset trees no longer match beyond this point. We also proceed down the right branch of the reference nodes and find no match, so our traversal of the two trees from these nodes is complete and we restart our search at the next pair of nodes.</p>
Cited by 0Open access