article · 2018

Query-Centric Failure Recovery for Distributed Stream Processing Engines

Cited by 0Abstract and details only

Abstract

Correlated failures that usually involve a number of nodes failing simultaneously have significant effect on systems' availability, especially for streaming applications that require real-time analysis. Most state-of-the-art distributed stream processing engines focus on recovering individual operator failure. By analyzing the existing recovery techniques, we identify the challenges and propose a fault-tolerance framework that can tolerate both individual and correlated failures with minimum overhead during the system's normal execution. Our progressive and query-centric recovery paradigm carefully schedules the recovery of failed operators based on the current availability of resources, such that the outputs of queries can be recovered as early as possible. We also formulate the new problem of recovery scheduling under correlated failures and design algorithms to optimize the recovery latency with a performance guarantee.

Cite this paper

APA 7

Su, L., & Zhou, Y. (2018). Query-Centric Failure Recovery for Distributed Stream Processing Engines. https://doi.org/10.1109/icde.2018.00129

MLA 9

Su, Li, and Yongluan Zhou. “Query-Centric Failure Recovery for Distributed Stream Processing Engines.” 2018, pp. 1276–79, https://doi.org/10.1109/icde.2018.00129.

Chicago

Su, Li, and Yongluan Zhou. 2018. “Query-Centric Failure Recovery for Distributed Stream Processing Engines.” https://doi.org/10.1109/icde.2018.00129.

Harvard

Su, L. and Zhou, Y. (2018) ‘Query-Centric Failure Recovery for Distributed Stream Processing Engines’. Available at: https://doi.org/10.1109/icde.2018.00129.

BibTeX

@article{su2018query,
  title = {{Query-Centric Failure Recovery for Distributed Stream Processing Engines}},
  author = {Su, Li and Zhou, Yongluan},
  pages = {1276--1279},
  year = {2018},
  doi = {10.1109/icde.2018.00129}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.