Nikhil Rasiwasia, Nuno Vasconcelos · 2008
Current image retrieval techniques have difficulties to retrieve images which exhibit distinct visual patterns but belong to the class of the query image. Previous attempts to improve generalization have shown that the introduction of semantic representations can mitigate this problem. We extend the existing query-by-semantic example (QBSE) retrieval paradigm by adding a second layer of semantic representation. At the first level, the representation is driven by patch-based visual features. Semantic concepts, from a predefined vocabulary, are modeled as Gaussian mixtures on a visual feature space, and images as vectors of posterior probabilities of containing each of the semantic concepts. At the second level, the representation is purely semantic. Semantic concepts are modeled as Dirichlet mixtures on the semantic feature space of QBSE, and images are again represented as vectors of posterior concept probabilities. It is shown that the proposed retrieval strategy, referred to as query-by-contextual-example (QBCE), is able to cope with the ambiguities of patch-based classification, exhibiting significantly better generalization than previous methods. An experimental evaluation on benchmark datasets shows that QBCE retrieval systems can substantially outperform their QBVE and QBSE counterparts, achieving high precision at most levels of recall.
Cited by 7
Liling Chen · K-State Research Exchange (Kansas State University) · 2011
Cited by 0
Nikhil Rasiwasia, Nuno Vasconcelos · 2008
In recent years, query-by-semantic-example (QBSE) has become a popular approach to do content based image retrieval. QBSE extends the well established query-by-example retrieval paradigm to the semantic domain. While various authors have pointed out the benefits of QBSE, there are still various open questions with respect to this paradigm. These include a lack of precise understanding of how the overall performance depends on various different parameters of the system. In this work, we present a systematic experimental study of the QBSE framework. This can be broadly divided into three categories. First, we examine the space of low-level visual features for its effects on the retrieval performance. Second, we study the space of learned semantic concepts, herein denoted as the ldquosemantic spacerdquo, and show that not all semantic concepts are equally informative for retrieval. Finally, we present a study of the intrinsic structure of the semantic space, by analyzing the contextual relationships between semantic concepts and show that this intrinsic structure is crucial for the performance improvements.
Cited by 11
Jonathan Ultis · 2001
This paper describes an implementation of query-by-example, or relevance feedback, for text. The implementation uses Google's search engine to perform a keyword query as requested by the user. If the user requires more information, the user may score documents in the result set as relevant or irrelevant. An implementation of the AdaBoost algorithm is then used choose words that separate the relevant documents from a random document set. Examples of negative document sets are also tested. An example query and refinements of the query is presented. The results seem promising. The system seems to propose new keywords that are sensible to the requested context. Many of the keywords prove useful in constructing new queries. However, refinement using exactly the new terms predicted by the system does not seem to return noticeably better or worse results. This may be the result of an inexact fit between the design of AdaBoost and the capabilities of Google as a back end engine. ...
Cited by 0
Byeonggeun Kim, Mingu Lee, Jinkyu Lee et al. · arXiv (Cornell University) · 2019
A keyword spotting (KWS) system determines the existence of, usually predefined, keyword in a continuous speech stream. This paper presents a query-by-example on-device KWS system which is user-specific. The proposed system consists of two main steps: query enrollment and testing. In query enrollment step, phonetic posteriors are output by a small-footprint automatic speech recognition model based on connectionist temporal classification. Using the phonetic-level posteriorgram, hypothesis graph of finite-state transducer (FST) is built, thus can enroll any keywords thus avoiding an out-of-vocabulary problem. In testing, a log-likelihood is scored for input audio using the FST. We propose a threshold prediction method while using the user-specific keyword hypothesis only. The system generates query-specific negatives by rearranging each query utterance in waveform. The threshold is decided based on the enrollment queries and generated negatives. We tested two keywords in English, and the proposed work shows promising performance while preserving simplicity.
Cited by 2Open access
I.C. Song, Myoung Ho Kim · 2009
Cited by 7
John Thomas, John D. Gould · 1975
Many different query systems have been proposed. One way to partition extant and hypothetical query systems is on the basis of how English-like they are. One approach for an easy-to-use query system is to allow the user to state a question in natural English. The system may then disambiguate the possible interpretations of this question on the basis of context or on the basis of feedback questions to the user. A second approach is to require the user to state his question in a formal language system, but one that uses an English-like grammar and vocabulary. IBM's Interactive Query Facility (IQF) and SEQUEL exemplify this type of approach. A third approach is to require the user to state his question in a formal language system that does not attempt to appear English-like. Zloof's Query By Example language is best described by this third approach. In this paper we demonstrate experimentally the ease and accuracy with which nonprogrammers learned and used this powerful Query By Example language.
Cited by 155
Yaodong Zhang · 2013
This thesis is motivated by the challenge of searching and extracting useful information from speech data in a completely unsupervised setting. In many real world speech processing problems, obtaining annotated data is not cost and time effective. We therefore ask how much can we learn from speech data without any transcription. To address this question, in this thesis, we chose the query-by-example spoken term detection as a specific scenario to demonstrate that this task can be done in the unsupervised setting without any annotations. To build the unsupervised spoken term detection framework, we contributed three main techniques to form a complete working flow. First, we present two posteriorgram-based speech representations which enable speaker-independent, and noisy spoken term matching. The feasibility and effectiveness of both posteriorgram features are demonstrated through a set of spoken term detection experiments on different datasets. Second, we show two lower-bounding based methods for Dynamic Time Warping (DTW) based pattern matching algorithms. Both algorithms
Cited by 0
N. Boujemaa, J. Fauqueur, Valérie Gouet · 2003
Over the last ten years, the crucial problem of information retrieval in multimedia documents has boosted research activities in the eld of visual appearance indexing and retrieval by content. In the early research years, the concept of the query by visual example (QBVE) has been proposed and shown to be relevant for visual information retrieval. It is obvious that QBVE is not able to satisfy the multiple visual search usage requirements. In this paper, we focus on two major approaches that correspond to two dierent retrieval paradigms. First, we present the partial visual query that ignores the background of the images and allows a straight user expression on its visual interest without relevance feedback mechanism. The second retrieval paradigm consists in searching for the user mental image when no starting visual example is available. This new approach relies on the unpervised generation of a visual thesaurus from which query by logical composition of region categories can be performed. This query paradigm is closely related to that of text retrieval. Mental image search is a challenging and promising issue for retrieval by visual content in the forthcoming years since it allows dierent rich user expression and interaction modes with the search engine.
Cited by 0
Abdullah Uz Tansel, M. Erol Arkun, Gültekin Özsoyoğlu · IEEE Transactions on Software Engineering · 1989
The authors propose a graphical query language, Time-by-Example (TBE), which has suitable constructs for interacting with historical relational databases in a natural way. TBE is user-friendly. It follows the graphical, two-dimensional approach of such previous languages as Query-by-Example (QBE), Aggregation-by-Example (ABE), and Summary-Table-by-Example (STBE). TBE also uses the hierarchical window (subquery) concept of ABE and STBE. TBE manipulates triple-valued (set-triple-valued) attributes and historical relations. Set-theoretic expressions are followed to deal with time intervals. The BNF specification for TBE is given.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Cited by 62
Sheikh Muhammad Sarwar, James Allan · 2020
We propose a Query by Example (QBE) setting for cross-lingual event retrieval. In this setting, a user describes a query event using example sentences in one language, and a retrieval system returns a ranked list of sentences that describe the query event, but from a corpus in a different language. One challenge in this setting is that a sentence may mention more than one event. Hence, matching the query sentence with document sentence results in a noisy matching. We propose a Semantic Role Labeling (SRL) based approach to identify event spans in sentences and use a state-of-the-art sentence matching model, Sentence BERT (SBERT) to match event spans in queries and documents without any supervision. To evaluate our approach we construct an event retrieval dataset from ACE which is an existing event detection dataset. Experimental results show that it is valuable to predict event spans in queries and documents and our proposed unsupervised approach achieves superior performance compared to Query Likelihood (QL), Relevance Model 3 (RM3) and SBERT.
Cited by 5
Flavio Xavier Ferreira, Daniela da Cruz, Pedro Rangel Henriques et al. · RepositóriUM (Universidade do Minho) · 2009
XML, as a general-purpose annotation system for creating custom markup languages, is becoming more and more important. XML annotations give structure to plain documents and help to interpret their content, making them human or machine readable. However, mechanisms to pretty-print those annotated documents or process them in order to extract information are crucial to make them useful. In a similar way, a collection of XML documents, without any tools capable of retrieving information from it, is useless. To search for specific elements in a marked up document we have, at least, two options: XPath and XQuery. However, the learning curve of these two dialects is high, requiring a considerable level of knowledge. In this context, the idea of Query-by-example can be an important contribution to make easier this learning process, freeing the user from knowing the specific query language details or even the document structure. In this paper, we describe our approach to QBE based on an sample document from the collection where the user specifies his needs. First, we focus on the choice of the adequate document to serve as a sample of the collection, based on different metrics computed over the document. Then, we discuss the query generation from the user needs' specifications, namely components (elements or attributes) selection and filtering.
Cited by 2Open access
Anup Singh, Demuynck, Kris, Arora, Vipul · Zenodo (CERN European Organization for Nuclear Research) · 2022
An ideal audio retrieval system efficiently and robustly recognizes a short query snippet from an extensive database. However, the performance of well-known audio fingerprinting systems falls short at high signal distortion levels. This paper presents an audio retrieval system that generates noise and reverberation robust audio fingerprints using the contrastive learning framework. Using these fingerprints, the method performs a comprehensive search to identify the query audio and precisely estimate its timestamp in the reference audio. Our framework involves training a CNN to maximize the similarity between pairs of embeddings extracted from clean audio and its corresponding distorted and time-shifted version. We employ a channel-wise spectral-temporal attention mechanism to capture salient time indices and spectral bands in the CNN features. The attention mechanism enables the CNN to better discriminate the audio by giving more weight to the salient spectral-temporal patches in the signal. Experimental results indicate that our system is efficient in computation and memory usage while being more accurate, particularly at higher distortion levels, than competing state-of-the-art systems and scalable to a larger database.
Cited by 0Open access
CINQUE, LUIGI, DE MARSICO, Maria, PELLICANÒ A. · IRIS Research product catalog (Sapienza University of Rome) · 2000
Retrieval of color images is performed by submitting a sketch of color distribution within target images
Cited by 0
Aloia N., Argentieri G. · ISTI Open Portal · 1980
No abstract available.
Cited by 0Open access
Dimitriou, Konstantinos · Zenodo (CERN European Organization for Nuclear Research) · 2014
In this thesis we investigate whether certain groups of features relate to the retrieval quality of Freesound. Freesound is an online collaborative audio database. People from different disci- plines share their own audio materials under the different types of Creative Commons licenses. Freesound’s similarity search functionality is one of the the features that is used by more users everyday. Freesound library is expanded everyday, as more users share their sounds, adding to the corpus of existing sounds. This expansion has brought to surface the need for a method that will be able to understand the similarity between sounds and retrieve the most relevant ones, given an example.
Cited by 0Open access
ประภาศรี บิดาศักดิ์ · NRCT Data Center
Cited by 0Open access
Emmanuel Debanne, Philippe Mulhem · Association en Recherche d’Information et Applications · 2004
RÉSUMÉ . Cet article décrit et défini l'utilisation de requêtes par l'exemple (QBE) dans le cadre de recherche symbolique d'images photographiques. La nouveauté de cette approche consiste en l'utilisation conjointe d'indexation symbolique automatique et d'un formalisme de représentation de connaissances pour représenter le contenu des images. De plus, le mécanisme d'abstraction perm la recherche d'images par l'exemple et le bouclage de pertinence basés sur la représentation symbolique des images, et pas sur leur description signal de bas niveau. Nous montrons sur deux collections d'images d'un total de plus de 1100 photographies que la recherche par l'exemple fournit des résultats comparables à ceux par symboles en terme de mesures de rappel-précision.
Cited by 1Open access
Sujuan Hou, Shangbo Zhou · Mathematical Problems in Engineering · 2013
Query by example video retrieval aims at automatic retrieval of video samples which are similar to a user-provided example from video database. Considering that much of prior work on video analysis support retrieval using only visual features, in this paper, a two-step method for query by example is proposed, in which both audio and visual features are used. In the proposed method, a set of audio and visual features are, respectively, extracted from the shot level and key frame level. Among these features, audio features are employed to rough retrieval, while visual features are applied to refine retrieval. The experimental results demonstrate the good performance of the proposed approach.
Cited by 5Open access
Frederico Valente (410099), Carlos Costa (410100), Augusto Silva (308822) · Figshare · 2015
Cited by 0Open access