Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

preprint · arXiv (Cornell University) · 2014

Enabling Incremental Query Re-Optimization

Mengmeng Liu, Zachary G. Ives, Boon Thau Loo

Cited by 1Open access

Abstract

As declarative query processing techniques expand in scope --- to the Web, data streams, network routers, and cloud platforms --- there is an increasing need for adaptive query processing techniques that can re-plan in the presence of failures or unanticipated performance changes. A status update on the data distributions or the compute nodes may have significant repercussions on the choice of which query plan should be running. Ideally, new system architectures would be able to make cost-based decisions about reallocating work, migrating data, etc., and react quickly as real-time status information becomes available. Existing cost-based query optimizers are not incremental in nature, and must be run "from scratch" upon each status or cost update. Hence, they generally result in adaptive schemes that can only react slowly to updates. An open question has been whether it is possible to build a cost-based re-optimization architecture for adaptive query processing in a streaming or repeated query execution environment, e.g., by incrementally updating optimizer state given new cost information. We show that this can be achieved beneficially, especially for stream processing workloads. Our techniques build upon the recently proposed approach of formulating query plan enumeration as a set of recursive datalog queries; we develop a variety of novel optimization approaches to ensure effective pruning in both static and incremental cases. We implement our solution within an existing research query processing system, and show that it effectively supports cost-based initial optimization as well as frequent adaptivity.

Cite this paper

APA 7

Liu, M., Ives, Z. G., & Loo, B. T. (2014). Enabling Incremental Query Re-Optimization. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1409.6288

MLA 9

Liu, Mengmeng, et al. “Enabling Incremental Query Re-Optimization.” arXiv (Cornell University), 2014, https://doi.org/10.48550/arxiv.1409.6288.

Chicago

Liu, Mengmeng, Zachary G. Ives, and Boon Thau Loo. 2014. “Enabling Incremental Query Re-Optimization.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1409.6288.

Harvard

Liu, M., Ives, Z.G. and Loo, B.T. (2014) ‘Enabling Incremental Query Re-Optimization’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.1409.6288.

BibTeX
@misc{liu2014enabling,
  title = {{Enabling Incremental Query Re-Optimization}},
  author = {Liu, Mengmeng and Ives, Zachary G. and Loo, Boon Thau},
  journal = {arXiv (Cornell University)},
  year = {2014},
  doi = {10.48550/arxiv.1409.6288}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.

Related papers