Ratan K. Ghosh, Hiranmay Ghosh · 2023
A many-core distributed system consists of multiple multi-core node clusters connected via network on chips (NoCs). Scaling up performance on a many-core system requires careful partitioning and placement of data to extract maximum parallelism. Furthermore, there are technological innovations with specialized hardware which accelerate performance. These include graphics processing unit (GPU), field programmable gateway arrays (FPGAs), and machine/deep learning (ML/DL) accelerators like Google's tensor processing unit (TPU). Simultaneously, intensive research on programming newer architecture led to the discovery of diverse computing abstractions such as MapReduce, Spark, and Dryad for data-parallel applications on these systems. These days, large-scale data-intensive applications like page rank, placement of ads, and social networks are implemented using many-core distributed systems. There is no single abstraction that fits all data-parallel applications. As a result, the applications performing well in one model perform poorly in another. The programmers find the context switching from one abstraction to another difficult, and their productivity suffers. Software distributed shared memory (S-DSM) research saw a resurgence with newer computing platforms. From a programmer's perspective, the shared memory programming model is a natural extension of the uniprocessor memory model on a distributed system. S-DSM implementation is transparent to the programmer. So, it allows the programmers to handle synchronizations in the familiar shared memory model. Their productivity is unhindered by the complexity of message-passing. The biggest challenge in S-DSM implementation is maintaining the consistency of shared variables. Inconsistent values of shared variables lead to a program's behavior being different from the programmer's expectations.
Cited by 5
Changhun Lee · 2002
Practical parallel machines which are parallel virtual machine (PVP), symmetric multiprocessor (SMP), massively parallel processor (MPP), distributed shared memory (DSM), and clusters of workstation (COWs) are described. According to the classification of practical parallel machines, this paper mainly deals with distributed shared memory (DSM). The concept of DSM is explained, and the comparison among software, hardware, and hybrid DSM implementations are presented.
Cited by 0
Alan Judge, Patrick Nixon, Brendan Tangney et al. · eCite Digital Repository (University of Tasmania) · 1999
Cited by 4Open access
John K. Bennett, J. B. Carter, Anna L. Cox et al. · 1992
Distributed shared memory (DSM) is the provision in software of a shared memory programming model on a distributed memory machine [6]. We are exploring the use of DSM in a cluster-based computing environment of workstations and servers connected by a local internetwork.We have concentrated so far on using DSM to program compute-intensive tasks on networks of workstations. For these applications, DSM is a superior programming model compared to message passing, because it relieves the programmer from having to worry about data motion. Second, DSM allows applications written for shared memory machines to be ported with relative ease to distributed memory machines. Finally, DSM offers a natural paradigm for integrating both the locally shared memory and the globally distributed memory of anticipated future networks of shared memory multiprocessors. In particular, in such an environment, local interprocess communication is likely to dominate and therefore should be implemented in the most efficient form possible, i.e., by using the (hardware) shared memory available. It then follows that to achieve a single paradigm for both local and remote interprocess communication, DSM is the natural choice.
Cited by 3Open access
Qiusheng Chen, Jennifer L. Welch, Qiusheng Chen · Distributed Computing · 2004
The Aleph Toolkit is a collection of Java packages that implements a simple, platform-independent dis tributed shared memory [MPH]. But currently it lacks of failure recovery mechanism and is not suitable for long-running distributed applications. The objective of this project is to make Aleph to be a fault-tolerant distributed shared memory computing platform.
Cited by 1Open access
Luis M. Silva, João Gabriel Silva · 1997
Distributed shared memory (DSM) is a very promising programming model for exploiting the parallelism of distributed memory systems, since it provides a higher level of abstraction than simple message passing. Although the nodes of standard distributed systems exhibit high crash rates only very few DSM environments have some kind of support for fault-tolerance. In this paper, we present a checkpointing mechanism for a DSM system that is efficient and portable. It offers some portability because it is built on top of MPI and uses only the services offered by MPI and a POSIX compliant local file system. As far as we know, this is the first real implementation of such a scheme for DSM. Along with the description of the algorithm we present experimental results obtained in a cluster of workstations. We hope that our research shows that efficient, transparent and portable checkpointing is viable for DSM systems. Keywords: Distributed Shared Memory, Checkpointing, Fault-Tolerance, Portabilit...
Cited by 0
Auld, Philip · W&M Publish (College of William & Mary) · 2001
Distributed shared memory (DSM) provides the illusion of shared memory processing to programs running on physically distributed systems. Many of these systems are connected by a broadcast medium network such as Ethernet. In this thesis, we develop a weakly coherent model for DSM that takes advantage of hardware-level broadcast. We define the broadcast DSM model (BDSM) to provide fine-grained sharing of user-defined locations. Additionally, since extremely weak DSM models are difficult to program, BDSM provides effective synchronization operations that allow it to function as a stronger memory. We show speedup results for a test suite of parallel programs and compare them to MPI versions.;To overcome the potential for message loss using broadcast on an Ethernet segment we have developed a reliable broadcast protocol, called Pipelined Broadcast Protocol (PBP). This protocol provides the illusion of a series of FIFO pipes among member process, on top of Ethernet broadcast operations. We discuss two versions of the PBP protocol and their implementations. Comparisons to TCP show the predicted benefits of using broadcast. PBP also shows strong throughput results, nearing the maximum of our 10Base-T hardware.;By combining weak DSM and hardware broadcast we developed a system that provides comparable performance to a common message-passing system, MPI. For our test programs that have all-to-all communication patterns, we actually see better performance than MPI. We show that using broadcast to perform DSM updates can be a viable alternative to message passing for parallel and distributed computation on a single Ethernet segment.
Cited by 1Open access
Andrew B. Hastings · 1992
Atomic transactions have proven to be an important technique for constructing reliable applications. Traditionally, transactions have been extended to distributed environments through the use of function shipping, a technique in which message passing or remote procedure calls are used to invoke computational requests on remote nodes. Recently, the data sharing approach to constructing distributed applications has received attention in the form of distributed file systems and distributed shared virtual memory. Applying the data sharing approach to transactions produces transactional distributed shared memory (TDSM) which yields benefits for a certain class of distributed application.
The union of transactions and distributed shared memory offers synergies in transaction recovery, concurrency control, and coherency control, but introduces challenges in transaction recovery. In this dissertation, I describe the design of a system that provides TDSM in the form of distributed recoverable virtual memory. Using the external pager interface of the Mach operating system, I implemented a prototype based on the Camelot distributed transaction facility. I analyze the prototype and its performance, offer techniques for improving the design of future TDSM systems, and characterize the applications for which TDSM is useful.
Cited by 5
Paulo Guedes, Miguel Castro · 2002
This paper describes the goals, programming model and design of DiSOM, a software based distributed shared memory system for a multicomputer composed of heterogeneous nodes connected by a high-speed network. A typical configuration is a cluster of tens of high-performance workstations and shared-memory multiprocessors of two or three different architectures, each with a processing power of a few hundred MIPS and several hundred kilobytes of memory, and connected by a high-speed interconnect such as ATM. Programs in DiSOM are written using a shared-memory multiprocessor model where synchronization objects are associated with data items. Programs use a threads library to start new threads, possibly at a specified node, and to synchronize between concurrent threads. Programs must call the synchronization primitives explicitly, as they would in a shared-memory multiprocessor. The system traps these calls and uses the information to drive both distributed synchronization and the memory coherence protocol. DiSOM uses the entry consistency memory model to ensure coherence. This model guarantees memory consistency, as long as an access to a data item is enclosed between an acquire and a release on the synchronization object associated with the data item. Stronger consistency models, such as release consistency and processor consistency, may also be supported.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Cited by 21
Sundarrajan S Kanthadai · OakTrust (Texas A&M University Libraries) · 1996
Due to the character of the original source materials and the nature of batch digitization, quality control issues may be present in this document. Please report any quality issues you encounter to digital@library.tamu.edu, referencing the URI of the item.
Cited by 12
A. Udaya Shankar · 2012
Cited by 0
Manish Parashar, Sumir Chandra · 2004
This chapter presents tools and environments for distributed shared memory (DSM), a software abstraction of shared memory on a distributed memory multiprocessor or cluster of workstations. It outlines the properties and features of DSM systems and classifies them based on their architectures. The chapter then describes cache coherence protocols for hardware based DSM systems including CC-NUMA, COMA, S-COMA, hybrid schemes including R-NUMA and AS-COMA, and hardware-based environments such as the MIT Alewife Machine and the Stanford FLASH multiprocessor. Finally, existing DSM system such as TreadMarks, Brazos, Mirage+, Orca, SAM, Midway, CRL, and fine-grained Shasta DSM are described.
Cited by 1
Krishna Kavi, Hyong-Shik Kim, Ben Lee et al. · Advances in computers · 2000
Cited by 4
Jackie Silcock, J. Silcock · 1995
Distributed Shared Memory is an important topic in distributed system research as it allows programmers to write and use code for shared memory multiprocessors on a system which does not in fact have physically shared memory. In this report I will discuss the literature I have read related to Distributed Shared Memory in an attempt to identify the important research issues in this area. The first section covers the high level design choices and research issues regarding DSM. The second section discusses the implementation issues and some current implementations of DSM. 1 Introduction Distributed Shared Memory (DSM) has been an active area of research since the mid 1980s. Some of the first work in this area was carried out by Libes [Libes1985], who implemented a shared variable scheme at user level on a Unix system. [Coulouris et al. 93] describes distributed shared memory as &quot;an abstraction used for sharing data between processes that do not share physical memory&quot;. Figure 1 shows th...
Cited by 0
Alokika Dash, Brian Demsky · ACM SIGPLAN Notices · 2009
We have developed a transaction-based approach to distributed shared memory(DSM) that supports object caching and generates path expression prefetches. A path expression specifies a path through the heap that traverses the objects to be prefetched. To our knowledge, this is the first prefetching approach that can prefetch objects whose addresses have not been computed or predicted. Our DSM uses both prefetching and caching of remote objects to hide network latency while relying on the two-phase transaction commit mechanism to preserve the simple transactional consistency model that we present to the developer. We have evaluated this approach on a matrix multiply benchmark. We have found that our approach enables to effectively utilize multiple machines in a cluster and also benefit from prefetching and caching of objects.
Cited by 0
Francisco de la Fuente, Felipe Torres, Fernando R. Rannou · 2012
Large-scale statistical reconstruction algorithms are known to be memory and processor intensive applications. For instance, the system matrix for a small animal scanner requires several gigabytes of memory storage and the algorithm usually needs many iterations to produce acceptable images. In this abstract we design distributed-shared memory (DSM) statistical reconstruction algorithms to exploit all available computational resources as a unified infrastructure and thereby improving the cost-efficiency of the investment and scalability of the system. We use and compare two distinct approaches. The first one uses the Unified Parallel C (UPC) compiler which transparently provides a global shared virtual address space across all computers. Data is physically stored in different computers, but threads can access any shared item as it if were in its local memory. The second approach combines OpenMP and Pthreads shared-memory libraries with the message-passing library MPI. In this case threads only have access to the node's local memory and access to remote data is carried out explicitly through message-passing. Early UPC experiments showed that keeping all data shared heavily affects reconstruction performance. Therefore, we devised a distribution method where some data is kept shared and other is kept private, mimicking somehow the library-based approach. However, even with data privatization, the compiler solution cannot compete with the library solutions. We explore three workload distribution strategies: LOR-based, Nonzero-based and Cores-based. The best performance is obtained with OpenMP+MPI and the Core-based balance algorithm, which reaches a speedup of 36 with 112 cores. However, both OpenMP+MPI and Pthreads+MPI outperform UPC by large. The low system efficiency of 0.32 is mainly due to the slow internode communication network.
Cited by 0
R. Ananthanarayanan · SMARTech Repository (Georgia Institute of Technology) · 1997
Software based Distributed Shared Memory (DSM) provides a desirable programming environment in workstation clusters. However, as compared to message-passing environments, DSM systems suffer from poor performance due to their lack of application specific knowledge. Broadly speaking, the inferior performance can be attributed to a mismatch in DSM coherence activities and application coherence requirements, and a mismatch in granularity of coherence and granularity of application data. The lack of cooperation between synchronization and coherence activities also leads to poor performance.
The main thesis of this dissertation is that explicit synchronization can be used to control memory coherence activities, and that application specific information can be efficiently captured by user provided annotations which associate data and synchronization operations. To prove this thesis, the dissertation refines data sharing patterns, and it identifies attributes to associate data and synchronization. A model is developed to represent shared data, explicit synchronization, and the attributes that associate data and synchronization.
The total cost of state-sharing in a DSM system includes the cost of synchronization. To this end, efficient algorithms are developed for synchronization primitives including semaphores, read-write locks, and barriers. These primitives capture a variety of interaction patterns, including mutual exclusion, producers-consumers, and readers-writers.
A set of cost metrics is developed to evaluate existing read-write lock algorithms. Since none of these algorithms provide satisfactory performance, a new algorithm is developed in which lock operations efficiently combine synchronization and memory coherence. Barriers are modified to allow association of data; these data barriers minimize the amount of data transfer in coherence activities, and avoid false sharing. A new abstraction called Queuing Semaphore is developed to capture producer-consumer relationships. Queuing semaphores integrate resource counting and memory coherence activities to reduce the cost of state-sharing.
Finally, the proposed algorithms were implemented in a prototype DSM system; for comparison, the prototype also supports message-passing primitives. Several applications were programmed both on top of the DSM system, and on top of the message-passing system. Empirical results show that the performance is essentially the same in both the DSM and the message-passing versions of the applications.
Cited by 0
Katrina Falkner, Henry Detmold, D. S. Munro et al. · 2003
There exists a wide spectrum of coherency models for use in distributed shared memory (DSM) systems. The choice of model for an application should ideally be based on the application's data access patterns and phase changes. However, in current systems, most, if not all of the parameters of the coherency model are fixed in the underlying DSM system. This forces the application either to structure its computations to suit the underlying model or to endure an inefficient coherency model. This paper introduces a unique approach to the provision of DSM based on the idea of compliance. Compliance allows an application to specify how the system should most effectively operate through a separation between mechanism, provided by the underlying system, and policy, pro-vided by the application. This is in direct contrast with the traditional view that an application must mold itself to the hard-wired choices that its operating platform has made. The contribution of this work is the definition and implementation of an architecture for compliant distributed coherency management. The efficacy of this architecture is illustrated through a worked example.
Cited by 3
Rasit Eskicioglu, T.A. Marsland · University of Alberta Library · 1996
Cited by 0Open access
Systems Architecture · 1993
A major problem with programming systems such as distributed memory multicomputers or networks of workstations has been the necessity for explicit, time-consuming and expensive message passing. Distributed shared memory enables such systems to appear to have a common memory though they may not physically share it. The Systems Architecture Research Centre at City University has worked on implementations of these kinds of systems and is experienced in the kinds of benefits they can offer. These are outlined for applications to fault tolerance. A DSM server implementation and its use are described. A method for using hardware to assist distributed shared memory is also described. Finally, a new architecture the Centre is studying is detailed. 1 Introduction The Systems Architecture Research Centre at City University (SARC) has been working at both the hardware and software levels on distributed memory multiprocessors for several years. Specifically, members of SARC have designed and impl...
Cited by 0