article · 2013

The Use of Locality Information on Data Intensive Parallel File Systems

Ricardo Ryoiti Sugawara, Líria Matsumoto Sato

Cited by 1Abstract and details only

Abstract

Many recent data intensive parallel systems builds with cost effective hardware and combine compute and storage facilities. Since bandwidth-bisecting networks are the norm, distributing jobs near data provides significant performance improvements. However, the data locality information is not easily available to the programmer. It requires interaction with file system internals, or the adoption of a custom programming and run-time frameworks that provide locality-aware job scheduling, such as Mapreduce and Hadoop. In this paper, we present a parallel file system implementation combined with virtual files presented as text that can be queried for locality data or written to control the placement of new data blocks. This simplifies how software that do not adhere to Mapreduce's model can benefit from computing near the data. To evaluate the proposed approach, a number of tests were run on an initial implementation using fast disks, with locality-aware cases showing from 2 to 9 times faster reads and higher processor utilization.

Cite this paper

APA 7

Sugawara, R. R., & Sato, L. M. (2013). The Use of Locality Information on Data Intensive Parallel File Systems. https://doi.org/10.1109/cse.2013.35

MLA 9

Sugawara, Ricardo Ryoiti, and Líria Matsumoto Sato. “The Use of Locality Information on Data Intensive Parallel File Systems.” vol. 4, 2013, pp. 167–73, https://doi.org/10.1109/cse.2013.35.

Chicago

Sugawara, Ricardo Ryoiti, and Líria Matsumoto Sato. 2013. “The Use of Locality Information on Data Intensive Parallel File Systems.” https://doi.org/10.1109/cse.2013.35.

Harvard

Sugawara, R.R. and Sato, L.M. (2013) ‘The Use of Locality Information on Data Intensive Parallel File Systems’. Available at: https://doi.org/10.1109/cse.2013.35.

BibTeX

@article{sugawara2013locality,
  title = {{The Use of Locality Information on Data Intensive Parallel File Systems}},
  author = {Sugawara, Ricardo Ryoiti and Sato, Líria Matsumoto},
  volume = {4},
  pages = {167--173},
  year = {2013},
  doi = {10.1109/cse.2013.35}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.