article · 2017

CDBB: An NVRAM-based Burst Buffer Coordination System for Parallel File Systems

Ziqi Fan, Fenggang Wu, Jim Diehl, David Hung-Chang Du, Doug Voigt

Cited by 1Abstract and details only

Abstract

For modern HPC systems, failures are treated as the norm instead of exceptions. To avoid rerunning applications from scratch, checkpoint/restart techniques are employed to periodically checkpoint intermediate data to parallel file systems. To increase HPC checkpointing speed, distributed burst buffers (DBB) have been proposed to use node-local NVRAM to absorb the bursty checkpoint data. However, without proper coordination, DBB is prone to suffer from low resource utilization. To solve this problem, we propose an NVRAM-based burst buffer coordination system, named collaborative distributed burst buffer (CDBB). CDBB coordinates all the available burst buffers, based on their priorities and states, to help overburdened burst buffers and maximize resource utilization. We built a proof-of-concept prototype and tested CDBB at the Minnesota Supercomputing Institute. Compared with a traditional DBB system, CDBB can speed up checkpointing by up to 8.4x under medium and heavy workloads and only introduces negligible overhead.

Cite this paper

APA 7

Fan, Z., Wu, F., Diehl, J., Du, D. H.-C., & Voigt, D. (2017). CDBB: An NVRAM-based Burst Buffer Coordination System for Parallel File Systems. https://doi.org/10.22360/springsim.2018.hpc.002

MLA 9

Fan, Ziqi, et al. “CDBB: An NVRAM-based Burst Buffer Coordination System for Parallel File Systems.” 2017, https://doi.org/10.22360/springsim.2018.hpc.002.

Chicago

Fan, Ziqi, Fenggang Wu, Jim Diehl, David Hung-Chang Du, and Doug Voigt. 2017. “CDBB: An NVRAM-based Burst Buffer Coordination System for Parallel File Systems.” https://doi.org/10.22360/springsim.2018.hpc.002.

Harvard

Fan, Z. et al. (2017) ‘CDBB: An NVRAM-based Burst Buffer Coordination System for Parallel File Systems’. Available at: https://doi.org/10.22360/springsim.2018.hpc.002.

BibTeX

@article{fan2017cdbb,
  title = {{CDBB: An NVRAM-based Burst Buffer Coordination System for Parallel File Systems}},
  author = {Fan, Ziqi and Wu, Fenggang and Diehl, Jim and Du, David Hung-Chang and Voigt, Doug},
  year = {2017},
  doi = {10.22360/springsim.2018.hpc.002}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.