Fiana Raiber, Oren Kurland · 2014
The query-performance prediction task has been described as estimating retrieval effectiveness in the absence of relevance judgments. The expectations throughout the years were that improved prediction techniques would translate to improved retrieval approaches. However, this has not yet happened. Herein we provide an in-depth analysis of why this is the case. To this end, we formalize the prediction task in the most general probabilistic terms. Using this formalism we draw novel connections between tasks --- and methods used to address these tasks --- in federated search, fusion-based retrieval, and query-performance prediction. Furthermore, using formal arguments we show that the ability to estimate the probability of effective retrieval with no relevance judgments (i.e., to predict performance) implies knowledge of how to perform effective retrieval. We also explain why the expectation that using previously proposed query-performance predictors would help to improve retrieval effectiveness was not realized. This is due to a misalignment with the actual goal for which these predictors were devised: ranking queries based on the presumed effectiveness of using them for retrieval over a corpus with a specific retrieval method. Focusing on this specific prediction task, namely query ranking by presumed effectiveness, we present a novel learning-to-rank-based approach that uses Markov Random Fields. The resultant prediction quality substantially transcends that of state-of-the-art predictors.
Cited by 56
Steve Cronen-Townsend, Yun Zhou, W. Bruce Croft · 2002
We develop a method for predicting query performance by computing the relative entropy between a query language model and the corresponding collection language model. The resulting clarity score measures the coherence of the language usage in documents whose models are likely to generate the query. We suggest that clarity scores measure the ambiguity of a query with respect to a collection of documents and show that they correlate positively with average precision in a variety of TREC test sets. Thus, the clarity score may be used to identify ineffective queries, on average, without relevance information. We develop an algorithm for automatically setting the clarity score threshold between predicted poorly-performing queries and acceptable queries and validate it using TREC data. In particular, we compare the automatic thresholds to optimum thresholds and also check how frequently results as good are achieved in sampling experiments that randomly assign queries to the two classes.
Cited by 663
Grant Fritchey · Apress eBooks · 2018
The previous chapter showed how to gather query performance metrics. This chapter will show how to consume those metrics to identify long-running or frequently called queries. Then I’ll go over the tools built into Management Studio so you can understand how a given query is performing. I’ll also spend a lot of time talking about using execution plans, which are your best view into the decisions made by the query optimizer.
Cited by 0
Ben He, Iadh Ounis · Information Systems · 2005
Cited by 143
Grant Fritchey · Apress eBooks · 2026
Query tuning is not easy.
Cited by 0
Umadevi Paila (434858), Brad A. Chapman (434859), Rory Kirchner (434860) et al. · Figshare · 2015
Cited by 0Open access
Apress eBooks · 2009
Managing query performance is another one of those subjects that requires an entire book to provide the coverage it deserves, especially when you dive into the execution plans. However, dealing with problem queries is an integral part of every DBA’s skill set. Luckily, Grant Fritchey and Sajal Dam have written SQL Server 2008 Query Performance Tuning Distilled (Apress, 2009), so this doesn’t have to be a 500-page chapter.
Cited by 0
Jason Brimhall, David H. Dye, Jonathan Gennick et al. · Apress eBooks · 2012
SQL Server query performance tuning and optimization requires a multilayered approach. The following are a few key factors that impact SQL Server query performance:
Cited by 1
Grant Fritchey · Apress eBooks · 2022
Query tuning is not easy.
Cited by 0
Anna Shtok, Oren Kurland, David Carmel · Lecture notes in computer science · 2009
Cited by 79
For Vocabulary-Supported Image, Julia Vogel, Bernt Schiele · 2002
In [1], we proposed a two-stage retrieval framework which makes not only performance characterization but also performance optimization manageable. There, the performance optimization focused on the second stage of the retrieval framework. In this paper, we extend the method to a full two-stage performance characterization and optimization. In our retrieval framework, the user specifies a high-level concept to be searched for, the size of the image region to be covered by the concept (e.g.``Search images with 30-50% of sky'') and an optimization option (e.g. ``maximum recall'', ``maximum precision'' or ``joint maximization of precision and recall''). For the detection of each concept such as sky, a multitude of concept detectors exist that perform differently. In order to reach optimum retrieval performance, the detector best satisfying the user query is selected and the information of the corresponding concept detector is processed and optimized. Besides the optimization procedure itself the paper discusses the generation of multiple detectors per semantic concept. In experiments, the advantage of joint compared to individual optimization of first and second stage is shown.
Cited by 0
Atakan Gelecek, Robin Koç, Pınar Karagöz et al. · DSpace VŠB-TUO (VŠB-TUO) · 2025
Cited by 0
Grant Fritchey · Apress eBooks · 2012
Query performance tuning remains an important part of today's database applications. Yes, hardware performance is constantly improving. Upgrades to SQL Server-especially to the optimizer, which helps determine how a query is executed, and the query engine, which executes the query-lead to better performance all on their own. Many systems are moving into the cloud where certain aspects of the systems are managed for you. Despite all this, query performance tuning remains a vital mechanism for improving the performance of your database management systems. The beauty of query performance tuning is that, in many cases, a small change to an index or a SQL query can result in a far more efficient application at a very low cost. In those cases, the increase in performance can be orders of magnitude better than that offered by an incrementally faster CPU or a slightly better optimizer.KeywordsHardware ResourceExecution PlanQuery OptimizerQuery ExecutionQuery PerformanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Cited by 0
Rakebul Hasan, Fabien Gandon · Lecture notes in computer science · 2014
Cited by 3Open access
Sajad Ebrahimi, Maryam Khodabakhsh, Negar Arabzadeh et al. · Lecture notes in computer science · 2024
Cited by 10
David Carmel, Elad Yom‐Tov · Synthesis lectures on information concepts, retrieval, and services · 2010
Cited by 1
Amin Bigdeli, Sajad Ebrahimi, Negar Arabzadeh et al. · ACM Transactions on Intelligent Systems and Technology · 2025
The varying performance of information retrieval (IR) methods, including state-of-the-art transformer-based neural retrievers, across diverse queries poses a significant challenge for achieving robust and reliable retrieval effectiveness. Query Performance Prediction (QPP) seeks to estimate the effectiveness of a retrieval method for individual queries, enabling adaptive strategies to improve retrieval outcomes, particularly for challenging queries. However, existing QPP approaches face fundamental challenges: pre-retrieval methods often rely on surface-level query features that fail to capture the nuanced relationship between queries and retrieval effectiveness, while post-retrieval methods depend heavily on the quality of retrieved documents, which can be unreliable for difficult queries. To this end, we propose the Query Space Distance-Based QPP ( QSD-QPP ) framework, which leverages the deterministic and consistent behavior of retrieval methods to estimate query performance by referencing historical queries with known effectiveness. The approach is motivated by the observation that semantically or syntactically similar queries often exhibit consistent retrieval performance, a property that can be exploited to make reliable predictions for unseen queries. QSD-QPP operates in two modes: (1) a lightweight pre-retrieval instantiation that dynamically constructs a query subspace based on embedding distances to interpolate the performance of proximate historical queries, and (2) an enriched post-retrieval instantiation that incorporates contextualized embeddings, document interactions, and historical query associations to enhance prediction accuracy. By utilizing large-scale contextualized embeddings derived from pre-trained language models, QSD-QPP efficiently identifies semantically similar queries and leverages their performance for robust predictions. By addressing the inherent limitations of prior approaches, QSD-QPP achieves a balanced trade-off between computational efficiency, prediction accuracy, and scalability. We evaluate QSD-QPP on four benchmark datasets, including MS MARCO Dev and TREC Deep Learning tracks (2019, 2020, and DL-Hard), demonstrating its superior accuracy and robustness compared to state-of-the-art baselines in both pre-retrieval and post-retrieval QPP tasks. To ensure reproducibility and encourage further research, we publicly release the implementation of our work.
Cited by 1Open access
Kihoon Lee · IEICE Transactions on Information and Systems · 2012
Considerable effort has been devoted to minimizing XPath queries under the assumption that the minimal query is faster than the original query. However, little attention has been paid to the validity of the assumption. In this paper, we provide a detailed analysis on the effectiveness of XPath query minimization and present an extensive experimental evaluation on the effectiveness using six publicly available XQuery engines. To the best of our knowledge, this is the first work done towards this objective. Experiments on real and synthetic data sets show that although the assumption is valid for some cases, the performance of the minimal query is often lower than or almost equal to that of the original query.
Cited by 0Open access
Apress eBooks · 2009
Query performance tuning is an important part of today’s database applications. Often you can achieve large savings in both time and money with proper query performance tuning. Hardware performance is constantly improving, which can lead to an attitude suggesting that other methods of performance tuning are no longer important. Upgrades to SQL Server—especially to the optimizer, which helps determine how a query is executed, and the query engine, which executes the query—lead to better performance all on their own. The beauty of query performance tuning is that, in many cases, a small change to an index or a SQL query can result in a far more efficient application at a very low cost.KeywordsHardware ResourceExecution PlanQuery OptimizerQuery PerformanceDatabase ApplicationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Cited by 0
The EXA4MIND Consortium · Zenodo (CERN European Organization for Nuclear Research) · 2025
The EXA4MIND project enables Extreme Data analytics, leveraging top-notch computing facilities and optimum data stores – from object stores over relational databases to NoSQL databases. EXA4MIND aims to facilitate data-flow connectivity between its database and object-storage deployments, computing systems at supercomputing centres, and the European data ecosystem (EUDAT, EOSC, European Data Spaces). In such demanding setting, using the databases efficiently and effectively is critical. Deliverable D3.3 presents how different types of databases and data storage systems are used in the Platform and Application Cases with a focus on improvement provided with the effective data management in the AQIS workflows. The obtained improvements are presented under two subtitles: (i) Functional Improvements, and (ii) Performance Improvements. The functional improvements describe the new functionalities provided to the Application Cases with the use of databases and database querying, whereas the performance improvements include the improvements obtained in comparison to previous status of the functionalities and services by the Application Cases with the integration of databases or tailoring the way databases are accessed and queried. The solutions we present are also applicable to a variety of other possible use cases.
Cited by 0Open access