Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 2,281 papers for “parallel file systems”

Parallel File Systems

Robert Ross, Philip Carns, D Metheny · International series in management science/operations research/International series in operations research & management science · 2009

Cited by 0

PARALLEL FILE SYSTEMS By

A. D. Mccxxiv, Comunità Europea, Fondo Sociale Europeo et al. · 2006

The undersigned hereby certify that they have read and recommend to the Faculty of Graduate Studies for acceptance a thesis entitled

Cited by 0

Comparison of Parallel File Systems

Pazdera, Michal · Brno University of Technology Digital Library (Brno University of Technology) · 2014

Cílem této práce bylo porovnání několika zástupců paralelních souborových systému. Zaměřuje se na výkonnost operací čtení/zápis v závislosti na různých typech zátěže a spolehlivost systémů, která sleduje odolnost proti výpadkům a ztrátě dat. První část práce je věnována studiu osmi nejrozšířenějších zástupců paralelních souborových systému. Z nich byly vybráni tři konkrétní systémy pro podrobnější zkoumání: Lustre, GlusterFS a CephFS. Za účelem jejich otestování byla navržena a implementována automatizovaná sada testovacích úloh. Vybrané systémy byly postupně nainstalovány na testovací hardware a otestovány pomocí připravené testovací sady. Naměřené výsledky byly popsány a vzájemně porovnány. Závěrečná část práce hodnotí vlastnosti zvolených systémů a jejich vhodnost pro konkrétní typy zátěže.

Cited by 0

Characterizing and Optimizing Parallel File Systems

Apratim Purakayastha, Apratim Purakayastha · 1996

High-performance parallel file systems are needed to satisfy tremendous I/O requirements of parallel scientific applications. The design of such parallel file systems depends on a comprehensive understanding of the expected workload, but so far there have been very few usage studies of multiprocessor file systems. In the first part of this dissertation, we attempt to fill this void by measuring a real file-system workload on a production parallel machine, namely the CM-5 at the National Center for Supercomputing Applications. We collect information about nearly every individual I/O request from the mix of jobs running on the machine. Analysis of the traces leads to various recommendations for design of future parallel file systems. Our usage study showed that writes to write-only files are a dominant part of the workload. Therefore, optimizing writes could have a significant impact on overall performance. In the second part of this dissertation, we propose ENWRICH, a compute-processor ...

Cited by 0

Scaling NFSv4 with Parallel File Systems

Dean Hildebrand, Peter Honeyman · Deep Blue (University of Michigan) · 2005

Large grid installations require global access to massive data stores. Parallel file systems give high throughput within a LAN, but cross-site data transfers lack seamless integration, security, and performance. The GridNFS project, aims to provide scalable, transparent, and secure data management as well as a scalable and agile name space. A key challenge in exporting a parallel file system with NFSv4 is to provide high performance without sacrificing consistency. This paper introduces extensions to the NFSv4 protocol to support parallel access. We implemented a prototype of our design and present experiments demonstrating its scalable architecture.

Cited by 0Open access

Scaling NFSv4 with parallel file systems

Dean Hildebrand, Peter Honeyman · 2005

Large grid installations require global access to massive data stores. Parallel file systems give high throughput within a LAN, but cross-site data transfers lack seamless integration, security, and performance. The GridNFS project, aims to provide scalable, transparent, and secure data management as well as a scalable and agile name space. A key challenge in exporting a parallel file system with NFSv4 is to provide high performance without sacrificing consistency. This paper introduces extensions to the NFSv4 protocol to support parallel access. We implemented a prototype of our design and present experiments demonstrating its scalable architecture.

Cited by 9

Parallel simulation of parallel file systems and I/O programs

Rajive Bagrodia, Stephen Docy, Andy Kahn · 1997

Efficient I/O implementations can have a significant impact on the performance of parallel applications. This paper describes the design and implementation of PIOSIM, a parallel simulation library for MPI-IO programs. The simulator can be used to predict the performance of existing MPI-IO programs as a function of architectural characteristics, caching algorithms, and alternative implementations of collective I/O operations. This paper describes the simulator and presents the results of a number of performance studies to evaluate the impact of the preceding factors on a set of MPI-IO benchmarks, including programs from the NAS benchmark suite.

Cited by 24Open access

Index File Partitioning in Parallel Database Systems

Byeong-soo Jeong, Edward Omiecinski · 1996

In a parallel database system, a table is often partitioned into multiple fragments and stored on different nodes in order to exploit I/O parallelism. Since using an index is typical for processing a database query, the problem of how to design the index for such partitioned tables can be a crucial performance factor in a parallel database. In terms of the index for partitioned tables, we can think of two alternatives, i.e., local index and partitioned global index. These alternative indexing schemes might display different performance characteristics for parallel query processing when the index is frequently used. In this paper, we examine the problem of how to partition an index file in a shared-nothing parallel database system. We present query processing strategies under alternative index file partitioning schemes. We also study the performance impact of alternative index file partitioning schemes by using simulation under different workloads. 1 Introduction In a parallel database...

Cited by 0

Integrating Theory And PracticeIn Parallel File Systems

Thomas H. Cormen, David Kotz · 1993

Several algorithms forparallel disk systems haveappeared in the literature recently, and they are asymptotically optimal in terms ofthe number of disk accesses. Scalable systems with parallel disks must be able to run these algorithms. We present for the rst time alist of capabilities that must be provided by the system to support these optimal algorithms: control over declustering, querying about the con guration, independent I/O, and turning o parity, le caching, and prefetching. We summarizerecent theoretical and empirical work that justi es the need for these capabilities. In addition, we sketch anorganization for a parallel le interface with low-level primitives and higher-level operations. 1

Cited by 0

PARALLEL COPYING TOOLS FOR DISTRIBUTED FILE SYSTEMS

Kevin Matthew Nuss · Scholar Works (Boise State University) · 2013

Kevin Matthew Nuss, and they evaluated their presentation and response to questions during the final oral examination. They found that the student passed the final oral examination.

Cited by 0Open access

Scalable security for petascale parallel file systems

Andrew W. Leung, Ethan L. Miller, Stephanie Jones · 2007

Petascale, high-performance file systems often hold sensitive data and thus require security, but authentication and authorization can dramatically reduce performance. Existing security solutions perform poorly in these environments because they cannot scale with the number of nodes, highly distributed data, and demanding workloads. To address these issues, we developed Maat, a security protocol designed to provide strong, scalable security to these systems. Maat introduces three new techniques. Extended capabilities limit the number of capabilities needed by allowing a capability to authorize I/O for any number of client-file pairs. Automatic Revocation uses short capability lifetimes to allow capability expiration to act as global revocation, while supporting non-revoked capability renewal. Secure Delegation allows clients to securely act on behalf of a group to open files and distribute access, facilitating secure joint computations. Experiments on the Maat prototype in the Ceph petascale file system show an overhead as little as 6--7%.

Cited by 40

Caching and Writeback Policies in Parallel File Systems

Carla Schlatter Ellis, David Kotz, David Kotz · 1993

Improvements in the processing speed of multiprocessors are outpacing improvements in the speed of disk hardware. Parallel disk I/O subsystems have been proposed as one way to close the gap between processor and disk speeds. Such parallel disk systems require parallel file system software to avoid performance-limiting bottlenecks. We discuss cache management techniques that can be used in a parallel file system implementation for multiprocessors with scientific workloads. We examine several writeback policies, and give results of experiments that test their performance. Kotz and Ellis: Caching and Writeback Policies in Parallel File Systems 3 Symbols used: ! % * @ Kotz and Ellis: Caching and Writeback Policies in Parallel File Systems 4 1 Introduction It is increasingly difficult to provide sufficient I/O bandwidth to keep parallel supercomputers running at full speed for large problems, which may consume and produce immense amounts of data. Recent trends have shown that improvem...

Cited by 0

Adding Data Management Services to Parallel File Systems

Scott Brandt · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2015

The objective of this project, called DAMASC for “Data Management in Scientific Computing”, is to coalesce data management with parallel file system management to present a declarative interface to scientists for managing, querying, and analyzing extremely large data sets efficiently and predictably. Managing extremely large data sets is a key challenge of exascale computing. The overhead, energy, and cost of moving massive volumes of data demand designs where computation is close to storage. In current architectures, compute/analysis clusters access data in a physically separate parallel file system and largely leave it scientist to reduce data movement. Over the past decades the high-end computing community has adopted middleware with multiple layers of abstractions and specialized file formats such as NetCDF-4 and HDF5. These abstractions provide a limited set of high-level data processing functions, but have inherent functionality and performance limitations: middleware that provides access to the highly structured contents of scientific data files stored in the (unstructured) file systems can only optimize to the extent that file system interfaces permit; the highly structured formats of these files often impedes native file system performance optimizations. We are developing Damasc, an enhanced high-performance file system with native rich data management services. Damasc will enable efficient queries and updates over files stored in their native byte-stream format while retaining the inherent performance of file system data storage via declarative queries and updates over views of underlying files. Damasc has four key benefits for the development of data-intensive scientific code: (1) applications can use important data-management services, such as declarative queries, views, and provenance tracking, that are currently available only within database systems; (2) the use of these services becomes easier, as they are provided within a familiar file-based ecosystem; (3) common optimizations, e.g., indexing and caching, are readily supported across several file formats, avoiding effort duplication; and (4) performance improves significantly, as data processing is integrated more tightly with data storage. Our key contributions are: SciHadoop which explores changes to MapReduce assumption by taking advantage of semantics of structured data while preserving MapReduce’s failure and resource management; DataMods which extends common abstractions of parallel file systems so they become programmable such that they can be extended to natively support a variety of data models and can be hooked into emerging distributed runtimes such as Stanford’s Legion; and Miso which combines Hadoop and relational data warehousing to minimize time to insight, taking into account the overhead of ingesting data into data warehousing.

Cited by 0Open access

DSFS: Decentralized security for large parallel file systems

Zhongying Niu, Hong Jiang, Ke Zhou et al. · 2010

This paper describes DSFS, a decentralized security system for large parallel file system. DSFS stores global access control lists (ACLs) in a centralized decision-making server and pushes pre-authorization lists (PALs) into storage devices. Thus DSFS allows users to flexibly set any access control policy for the global ACL or even change the global ACL system without having to upgrade the security code in their storage devices. With pre-authorization lists, DSFS enables a network-attached storage device to immediately authorize I/O, instead of demanding a client to acquire an authorization from a centralized authorization server at a crucial time. The client needs to acquire only an identity key from an authentication server to access any devices she wants. Experimental results show that DSFS achieves higher performance and scalability than traditional capability-based security protocols.

Cited by 2

Doctoral Committee: Distributed Access to Parallel File Systems

Dean Hildebrand · 2007

To my family… individually they exceed my wildest expectations, together they give me strength to climb the highest mountains. ii ACKNOWLEDGEMENTS This dissertation is a testament to the dedication and abilities of many people. Their support and guidance transformed me into who I am today. My Ph.D. advisor Peter Honeyman is the main pillar of this dissertation. His insights and criticisms taught me the fundamental elements of successful research. He has a true love of computer science, which transfers to everyone around him. The other members of my committee, Farnam Jahanian, Bill Martin, Pete Chen, and Darrell Long, made many helpful suggestions at my proposal that helped guide the focus of this dissertation. My work depends heavily upon many bleeding edge technologies, each of which would not exist without the dedication of many brilliant and talented people. Garth Gibson, Lee Ward, and Gary Grider in particular championed pNFS to the wider storage and high-performance community, sparking interest for its continued research. The IETF pNFS working group, with all their bluster, raised many critical requirements and issues. pNFS would still be an unfinished IETF specification without the support and tireless efforts put forth by Marc Eshel at IBM and Rob Ross, Rob Latham, Murali Vilayannur, and the entire PVFS2 development team. In addition, I cannot forget Andy Adamson, Bruce Fields, Trond Myklebust, Olga Kornievskaia, David Richter, Jim Rees, and everyone else at CITI that have transformed Linux NFSv4 into the best distributed file system in existence.

Cited by 0