2008
With the advent of 64-bit processors, large main memories are set to become very common. This in turn translates to larger buffer pool configurations in database servers. Query optimizers however, currently assume all data is disk resident while optimizing queries. This assumption will no longer be valid when buffer pools become 100’s of gigabytes in size. In this paper we examine how data presence in the buffer pool can affect the choice of query plans in an optimizer. We examine the possible benefits of buffer-pool aware query optimization and propose a generic architecture for implementing such an optimizer. 1.
Cited by 0
Abdur Rafay · arXiv (Cornell University) · 2016
SQL-on-Hadoop systems, query optimization, data distribution over multiple nodes and parallelization techniques are few of the areas under extreme research these days. Big names like Amazon, Google, Microsoft and many more are working on implementing systems for faster access of data from multiple nodes reducing data mobility and increasing the parallelization. Queries are retrieved and reviewed by the database systems in an efficient way in the least amount of time by the introduction of various parallelization techniques by running the same query in parallel over different nodes carrying the data. Apart from multi-threading parallelization, there is another way of parallelization that can be performed in order to further reduce retrieval time hence improving the efficiency of the system; parallelization on user queries on top of a DBMS/RDBMS. In this paper, we will study one such technique of how multiple queries can run simultaneously on a system in order to increase the efficiency by reducing the data retrieval from the storage. Maximum sharing of workload has been performed by generating optimal and ubiquitous join plans for a set of queries and then fed them to GLADE (Generalized Linear Aggregate Distribution Engine), a scalable distributed system for large scale data analytics. Our main work is centered on generating GLADE join plans for a Multi-Query satisfying maximum number of queries in order to maximize data sharing and minimize data retrieval for each individual query.
Cited by 0Open access
Mohamed Mounir Hassan, Ahmed Mohammed Sultan · 2015
Semantic query optimization uses semantic knowledge in databases to rewrite queries and logic programs for the purpose of more efficient query evaluation. There has been a large body of work in the area of semantic query optimization. But, unfortunately, till now no commercial application of sematic query optimization techniques has received wide attention. In this paper, we address this problem by developing a unified framework (Application Programming Interface) called SQOPI that could be used by any application developer to semantically optimize queries executed against relational database regardless of DBMS type used. Our results show that SQOPI improves both time and I/O efficiency.
Cited by 0
Abdelkader Hameurlain, Franck Morvan · Lecture notes in computer science · 2009
Cited by 29
Kumail Saifuddin Saif · Zenodo (CERN European Organization for Nuclear Research) · 2020
Query performance is a key aspect for any development done in SAP HANA and it is very common to have performance issues when the data size grows in the system. SAP HANA offers effective tools and techniques for developers to make sure the query is optimized as per the hardware of SAP HANA database. Understanding the details of Query processing by different HANA engines like Calculation engine, Join engine, OLAP engine to name a few, and the role of the SQL optimizer, SQL plan cache, PlanViz, OOM dumps helps developers to write and test the code to make sure performance is optimized.
Cited by 0Open access
Ning Zhang · Lecture notes in computer science · 2004
Cited by 5
Xin Zhang, Ahmed Eldawy · Proceedings of the VLDB Endowment · 2024
Query optimization is a key component in database management systems (DBMS) and distributed data processing platforms. Recent research in the database community incorporated techniques from artificial intelligence to enhance query optimization. Various learning models have been extended and applied to the query optimization tasks, including query execution plan, query rewriting, and cost estimation. The tasks involved in query optimization differ based on the type of data being processed, such as relational data or spatial geometries. This tutorial reviews recent learning-based approaches for spatial query optimization tasks. We go over methods designed specifically for spatial data, as well as solutions proposed for high-dimensional data. Additionally, we present learning-based spatial indexing and spatial partitioning methods, which are also vital components in spatial data processing. We also identify several open research problems in these fields.
Cited by 2
Nilesh Dalvi, Sumit Sanghai, Prasan Roy et al. · Journal of Computer and System Sciences · 2003
Cited by 59
Karl Aberer, Gisela Fischer · 1995
Although the main difference between the relational and the object-oriented data model is the possibility to define object behavior, query optimization techniques in objectoriented database systems are mainly based on the structural part of objects. We claim that the optimization potential emerging from methods has been strongly underestimated so far. In this paper we concentrate on the question of how semantic knowledge about methods can be considered in query optimization. We rely on the algebraic and rulebased approach for query optimization and present a framework that allows to integrate schema-specific knowledge by tailoring the query optimizer according to the particular application's needs. We sketch an implementation of our concepts within the OODBMS VODAK using the Volcano optimizer generator.
Cited by 0
Dongfeng Chen, Rada Chirkova, Fereidoon Sadri et al. · Acta Informatica · 2013
Cited by 12
収兄 宮崎 · Medical Entomology and Zoology · 1990
Cited by 4
University Research Chair · Encyclopedia of Database Systems · 2009
Cited by 0
Elsidani Elariss, Haifa, Khaddaj, Souheil, Greenhill, Darrel · Research Repository (Kingston University London) · 2009
Cited by 0
Xiaoyi Zhou, Jie Luo, Tao He · 2017
As the main query language on RDF data, SPARQL constantly drawing attention from the Semantic Web communities. In this paper we present a mechanism that can identify common sub queries in multiple RDF queries and eliminate redundant sub queries, significantly improving the efficiency of executing multiple RDF queries, both simultaneously and sequentially. Unlike previous optimization works concentrated on optimizing a single RDF query, our method focus on improving the performance of multiple similar RDF queries, which commonly occur in real world applications related to RDF data. Extensive experiments over benchmark RDF repositories of millions of triples confirm that our method is superior to the state-of-the-art methods when common sub queries exist in multiple SPARQL queries.
Cited by 0
K. O'Gorman, D. Agrawal, A. El Abbadi · 2001
The multiple-query optimization (MQO) problem has been well-studied in the research literature, usually by means of identifying and exploiting the occurence of common subexpressions, and has required implementation in the database engine. Observing that common subexpressions derive from common data, and that the amount of data is usually greatest at the source, we propose an optimization technique that exploits the presence of sharable access patterns to underlying data, especially scans of large portions of tables or indexes, in environments where query queueing or batching is an acceptable approach. We show that simultaneous queries with such sharable accesses have a tendency to form synchronous groups (teams) which bene t each other through the operation of the disk cache, in eect using it as an implicit pipeline. We propose a novel method of optimization, exploiting this tendency by scheduling the queries to enhance this tendency, and show that this can be accomplished even from outside the database engine with application server middleware. We present an algorithm for scheduling from a queue of similar queries, designed to promote such teamwork. This is implemented as middleware for use with a commercial database engine. Finally, we present tests using the query mix from the TPC-R benchmark, achieving a speedup of 2.34 over the default scheduling provided by the database.
Cited by 0
Haifa Elsidani Elariss, Souheil Khaddaj, D. Greenhill · 2009
Cited by 3Open access
Hyoseop Shin, Minsoo Lee · Lecture notes in computer science · 2005
Cited by 0
Avani Shastri · Digital WPI · 2011
A continuous top-k query retrieves the k most preferred objects from a data stream according to a given preference function. These queries are important for a broad spectrum of applications from web-based advertising, network traffic monitoring, to financial analysis. Given the nature of such applications, a data stream may be subjected at any given time to multiple top-k queries with varying parameter settings requested simultaneously by different users. This workload of simultaneous top-k queries must be executed efficiently to assure real time responsiveness. However, existing methods in the literature focus on optimizing single top-k query processing, thus would handle each query independently. They are thus not suitable for handling large numbers of such simultaneous top-k queries due to their unsustainable resource demands. In this thesis, we present a comprehensive framework, called MTopS for Multiple Top-K Optimized Processing System. MTopS achieves resource sharing at the query level by analyzing parameter settings of all queries in the workload, including window-specific parameters and top-k parameters. We further optimize the shared processing by identifying the minimal object set from the data stream that is both necessary and sufficient for top-k monitoring of all queries in the workload. Within this framework, we design the MTopBand algorithm that maintains the up-to-date top-k result set in the size of O (k), where k is the required top-k result set, eliminating the need for any recomputation. To overcome the overhead caused by MTopBand to maintain replicas of the top-k result set across sliding windows, we optimize this algorithm further by integrating these views into one integrated structure, called MTopList. Our associated top-k maintenance algorithm, also called MTopList algorithm, is able to maintain this linear integrated structure, thus able to efficiently answer all queries in the workload. MTopList is shown to be memory optimal because it maintains only the distinct objects that are part of top-k results of at least one query. Our experimental study, using real data streams from domains of stock trades and moving object monitoring, demonstrates that both the efficiency and scalability in the query workload of our proposed technique is superior to the state-of-the-art solutions.
Cited by 0Open access
Purdue E-pubs, Rimma V. Nehme, Karen Works et al. · 2008
Abstract — In most database systems, traditional and stream systems alike, the optimizer picks a single query plan for all data based on the overall statistics of the data. It has however been repeatedly observed that real-life datasets are non-uniform. Selecting a single execution plan may result in a query execution that is ineffective for possibly large portions of the actual data. In this paper, we present a practical alternative to the current state-of-the-art query optimization techniques, termed a multi-route query mesh model (or short QM). The main idea of QM is to compute multiple routes (query plans), each designed for a particular subset of data with distinct statistical properties. Based on the execution routes and the data characteristics, a classifier model is induced. The classifier is used for efficient partitioning of the new data to assign the best route for query processing. We formulate the QM search space and analyze its
Cited by 0
Sheetal S. Dhande, Bamnote G.R · 2014
This paper is based on relatively newer approach for query optimization in object databases, which uses query decomposition and cached query results to improve execution a query. Issues that are focused here is fast retrieval and high reuse of cached queries, Decompose Query into Sub query, Decomposition of complex queries into smaller for fast retrieval of result.
Cited by 3Open access