Simo Juvaste, Simo Juvaste · 1997
: We present a new experimental framework, the F-PRAM model, for modeling and studying the effects of different properties of parallel machines. The goal of the new model is to provide programmer-level facilities for more accurate and portable computation-communication balancing. As a computation model, F-PRAM provides shared memory but guides the usage of the shared memory by a set of parameters. In comparison with the BSP model, we provide more asynchronous execution and model the costs and delays induced by communication bandwidth restrictions, communication latency, overhead, and synchronization independently of each other. We have written a high level language compiler and a parametrized emulator to measure the effects of different properties of parallel machines on different parallel algorithms. Keywords: F-PRAM, Parallel, Programming, Computation, Model, Compiler, Emulator 1 Introduction Most of the current research on parallel computing is based either on simplified models of...
Cited by 0
M.M. Eshaghian · 2003
Present a novel parallel programming model called Cluster-M. This model provides an environment for efficiently designing highly parallel machine independent software. Cluster-M is a generic model consisting of a collection of clusters such that each cluster represents a set of processors having similar level of interconnectivity. For every multiprocessor architecture there exists a Cluster-M representation. This leads to a mapping mechanism which allows a single software to be ported among various parallel computing systems.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Cited by 8
Lajos Schrettner · 2000
Cited by 0
Lawrence Snyder · DIMACS series in discrete mathematics and theoretical computer science · 1994
Cited by 2
Lawrence A. Crowl · 1988
Current parallel programming languages support only a narrow range of programming styles; force programmers to bind the specification of parallelism too early in program development; and limit the class of architectures for which they are effective. This paper presents a model for parallel programming based on uniform encapsulation and control, concurrent operations on objects, synchronous invocation with early reply, references to code in an environment, and a copy model of parameters and variables. This simple, coherent model enables programmers to use different styles of programming and to bind parallelism late in program development. In addition, the model may be efficiently implemented on a wide range of multiprocessor architectures. 1 Introduction Current imperative languages for parallel programming tend to provide separate facilities for parallel programming on top of a base sequential model of programming. These facilities typically include a distinct notion of process, monit...
Cited by 0
J. M. PÉREZ · 2015
Designing parallel codes is hard. One of the most important roadblocks to parallel programming is the presence of data dependencies. These restrict parallelism and, in general, to work them around requires complex analysis and leads to convoluted solutions that decrease the quality of the code. This thesis proposes a solution to parallel programming that incorporates data dependencies into the model. The programming model can handle that information and to dynamically find parallelism that otherwise would be hard to find. This approach improves both programmability and parallelism, and thus performance. While this problem has already been solved in OpenMP 4 at the time of this publication, this research begun before the problem was even being considered for OpenMP 3. In fact, some of the contributions of this thesis have had an influence on the approach taken in OpenMP 4. However, the contributions go beyond that and cover aspects that have not been considered yet in OpenMP 4. The approach we propose is based on function-level dependencies across disjoint blocks of contiguous memory. While finding dependencies under those constraints is simple, it is much harder to do so over strided and possibly partially overlapping sets of data. This thesis also proposes a solution to this problem. By doing so, we increase the range of applicability of the original solution and increase the span of applicability of the programming model. OpenMP4 does not currently cover this aspect. Finally, we present a solution to take advantage of the performance characteristics of Non-Uniform Memory Access architectures. Our proposal is at the programming model level and does not require changes in the code. It automatically distributes the data and does not rely on data migration nor replication. Instead, it is based exclusively on scheduling the computations. While this process is automatic, it can be tuned through minor changes in the code that do not require any change in the programming model. Throughout the thesis, we demonstrate the effectiveness of the proposal through benchmarks that are either hard to program using other paradigms or that have different solutions. In most cases, our solutions perform either on par or better than already existing solutions. This includes the implementations available in well-known high-performance parallel libraries. Dissenyar codis paral·lels es complex. Un dels principals esculls a l'hora de programar aplicacions paral·leles és la presència de dependències. Aquestes constrenyen el paral·lelisme, i en general, per evitar-les es requereix realitzar anàlisis complicades que donen lloc a solucions complexes que redueixen la qualitat del codi. Aquesta tesi proposa una solució a la programació paral·lela que incorpora al model les dependències de dades. El model de programació és capaç d'utilitzar aquesta informació per a trobar paral·lelisme que altrament seria molt difícil de detectar i d'extreure. Aquest enfoc augmenta la programabilitat i el paral·lelisme, i per tant també el rendiment. Tot i que al moment de la publicació d'aquesta tesi, el problema ja ha estat resolt a OpenMP 4, la recerca d'aquesta tesi va començar abans de que el problema s'hagués plantejat en l'àmbit d'OpenMP 3. De fet, algunes de les contribucions de la tesi han influït en la solució emprada a OpenMP 4. Tanmateix, les contribucions van més enllà i cobreixen aspectes que encara no han estat considerats a OpenMP 4. La proposta es basa en dependències a nivell de funció entre blocs de memòria continus i sense intersecció. Tot i que trobar dependències sota aquestes condicions és senzill, fer-ho sobre dades no contínues amb possibles interseccions parcials és molt més complex. Aquesta tesi també proposa una solució a aquest problema. Fent això, es millora el rang d'aplicació de la solució original i per tant el del model de programació. Aquest és un dels aspectes que encara no es contemplen a OpenMP 4. Finalment, es presenta una solució que té en compte les característiques de rendiment de les arquitectures NUMA (Accés No Uniforme a la Memòria). La proposta es planteja a nivell del model de programació i no precisa de canvis al codi ja que les dades es distribueixen automàticament. En lloc de basar-se en la migració i la replicació de les dades, es basa exclusivament en la planificació de l'execució de les computacions. Tot i que aquest procés és automàtic, es pot afinar mitjançant petits canvis en el codi que no arriben a alterar el model de programació. Al llarg d'aquesta tesi es demostra la efectivitat de les propostes a través de bancs de proves que son difícils de programar amb altres paradigmes o que tenen solucions diferents. A la majoria dels casos les nostres solucions tenen un rendiment similar o millor que les solucions preexistents, que inclouen implementacions en ben reconegues biblioteques paral·leles d'alt rendiment.
Cited by 3Open access
S Maclean, H Glaser, D DeRoure · 1998
A parallel programming model called the Pool Model is described. The Pool Model supports a parallel programming paradigm which is independent of shared memory threaded programs and of message-based network programming. The pool is merely an environment for constructing concurrent computations. The runtime system of the environment is sketched; highlighted are task location constraints and task racing.
Cited by 0
Marjanović, Vladimir · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · 2016
Even today supercomputing systems have already reached millions of cores for a single machine, which are connected by using a complex network interconnection. Reducing communication time across processes becomes the most important issue in order to achieve the highest possible performance. The Message Passing Interface (MPI), which is the most widely used programming model for large distributed memory, supports asynchronous communication primitives for overlapping communication and computation. However, these primitives are difficult to use and increase code complexity.\n\t\t\t\t which then requiring more development effort and making less readable programs.\n\t\t\t\t This thesis presents a new programming model, which allows the programmer to easily introduce the asynchrony necessary to overlap communication and computation. The proposed programming model is based on MPI and tasked based shared memory framework, namely OmpSs. The thesis further describes implementation details which in order to allow efficient inter-operation of the OmpSs runtime and MPI. \n\t\t\t\t The thesis demonstrates the hybrid use of MPI/OmpSs with several applications of which the HPL benchmark is the most important case study. The hybrid MPI/OmpSs versions significantly improve the performance of the applications compared with their pure MPI counterparts. For the HPL we get close to the asymptotic performance at relatively small problem sizes and still get significant benefits at large problem sizes. In addition, the hybrid MPI/OmpSs approach substantially reduces code complexity and is less sensitive to network bandwidth and operating system noise than the pure MPI versions.\n\t\t\t\t In addition, the thesis analyzes and compares current techniques for overlapping computation and collective communication, including approaches using point-to-point communications and additional communication threads, respectively. The thesis stresses the importance of understanding the characteristic of a computational kernel that runs concurrently with communication. Experimental evaluations is done using the Communication Computation Concurrent (CCUBE) synthetic benchmark, developed in this thesis, as well as the HPL.
Cited by 0
Cui Hui · 2013
Task parallel programming model is a widely used parallel programming model on multi-core platforms.With the intention of simplifying parallel programming and improving the utilization of multiple cores,this paper provides an introduction to the essential programming interfaces and the supporting mechanism used in task parallel programming models and discusses issues and the latest achievements from three perspectives: Parallelism expression,data management and task scheduling.In the end,some future trends in this area are discussed.
Cited by 5
Pérez Cáncer, Josep Maria · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · 2015
Designing parallel codes is hard. One of the most important roadblocks to parallel programming is the presence of data dependencies. These restrict parallelism and, in general, to work them around requires complex analysis and leads to convoluted solutions that decrease the quality of the code.\n\t\t\t\t This thesis proposes a solution to parallel programming that incorporates data dependencies into the model. The programming model can handle that information and to dynamically find parallelism that otherwise would be hard to find. This approach improves both programmability and parallelism, and thus performance.\n\t\t\t\t While this problem has already been solved in OpenMP 4 at the time of this publication, this research begun before the problem was even being considered for OpenMP 3. In fact, some of the contributions of this thesis have had an influence on the approach taken in OpenMP 4. However, the contributions go beyond that and cover aspects that have not been considered yet in OpenMP 4.\n\t\t\t\t The approach we propose is based on function-level dependencies across disjoint blocks of contiguous memory. While finding dependencies under those constraints is simple, it is much harder to do so over strided and possibly partially overlapping sets of data. This thesis also proposes a solution to this problem. By doing so, we increase the range of applicability of the original solution and increase the span of applicability of the programming model. OpenMP4 does not currently cover this aspect.\n\t\t\t\t Finally, we present a solution to take advantage of the performance characteristics of Non-Uniform Memory Access architectures. Our proposal is at the programming model level and does not require changes in the code. It automatically distributes the data and does not rely on data migration nor replication. Instead, it is based exclusively on scheduling the computations. While this process is automatic, it can be tuned through minor changes in the code that do not require any change in the programming model.\n\t\t\t\t Throughout the thesis, we demonstrate the effectiveness of the proposal through benchmarks that are either hard to program using other paradigms or that have different solutions. In most cases, our solutions perform either on par or better than already existing solutions. This includes the implementations available in well-known high-performance parallel libraries.
Cited by 0Open access
Dorit Naishlos, Joseph Nuzman, Chau‐Wen Tseng et al. · 2005
Explicit-multithreading (XMT) is a parallel programming model designed for exploiting on-chip parallelism. Its features include a simple thread execution model and an efficient prefix-sum instruction for synchronizing shared data accesses. By taking advantage of low-overhead parallel threads and high on-chip memory bandwidth, the XMT model tries to reduce the burden on programmers by obviating the need for explicit task assignment and thread coarsening. This paper presents features of the XMT programming model, and evaluates their utility through experiments on a prototype XMT compiler and architecture simulator. We find the lack of explicit task assignment has slight effects on performance for the XMT architecture. Despite low thread overhead, thread coarsening is still necessary to some extent, but can usually be automatically applied by the XMT compiler. The prefix-sum instruction provides more scalable synchronization than traditional locks, and the simple run-untilcompletion thread execution model (no busy-waits) does not impair performance. Finally, the combination of features in XMT can encourage simpler parallel algorithms that may be more efficient than more traditional complex approaches.
Cited by 0
Vladimir Marjanović · 2016
Even today supercomputing systems have already reached millions of cores for a single machine, which are connected by using a complex network interconnection. Reducing communication time across processes becomes the most important issue in order to achieve the highest possible performance. The Message Passing Interface (MPI), which is the most widely used programming model for large distributed memory, supports asynchronous communication primitives for overlapping communication and computation. However, these primitives are difficult to use and increase code complexity. which then requiring more development effort and making less readable programs. This thesis presents a new programming model, which allows the programmer to easily introduce the asynchrony necessary to overlap communication and computation. The proposed programming model is based on MPI and tasked based shared memory framework, namely OmpSs. The thesis further describes implementation details which in order to allow efficient inter-operation of the OmpSs runtime and MPI. The thesis demonstrates the hybrid use of MPI/OmpSs with several applications of which the HPL benchmark is the most important case study. The hybrid MPI/OmpSs versions significantly improve the performance of the applications compared with their pure MPI counterparts. For the HPL we get close to the asymptotic performance at relatively small problem sizes and still get significant benefits at large problem sizes. In addition, the hybrid MPI/OmpSs approach substantially reduces code complexity and is less sensitive to network bandwidth and operating system noise than the pure MPI versions. In addition, the thesis analyzes and compares current techniques for overlapping computation and collective communication, including approaches using point-to-point communications and additional communication threads, respectively. The thesis stresses the importance of understanding the characteristic of a computational kernel that runs concurrently with communication. Experimental evaluations is done using the Communication Computation Concurrent (CCUBE) synthetic benchmark, developed in this thesis, as well as the HPL. Las supercomputadoras están formadas por un creciente número de núcleos, del orden de millones en la actualidad, que se comunican a través de una compleja red de interconexión. Para obtener el más alto rendimiento posible es necesario reducir el tiempo de comunicación entre procesos. MPI ("Message Passing Interface", Interfaz de Paso de Mensajes), el modelo de programación más usado para grandes sistemas con memoria distribuida, permite llamadas de comunicación asíncrona para solapar la comunicación y la computación. Sin embargo, dichas llamadas son difíciles de usar e incrementan la complejidad del código, necesitándose un mayor esfuerzo en la implementación del código y dando lugar a programas más difíciles de leer. Esta tesis presenta un nuevo modelo de programación que permite al programador introducir fácilmente la asincronía necesaria para solapar la comunicación y la computación. El modelo de programación propuesto está fundamentado en MPI y la infraestructura basada en tareas y memoria compartida OmpSs. La tesis describe en profundidad los detalles de la implementación para la eficiente interoperabilidad entre OmpSs y MPI. En la tesis se demuestra el uso híbrido de MPI/OmpSs con distintas aplicaciones de las cuales el benchmark HPL es el más importante. La versión híbrida MPI/OmpSs mejora significativamente el rendimiento de las aplicaciones respecto a las versiones MPI originales. En el caso de HPL se acerca a un rendimiento asintótico para problemas relativamente pequeños, obteniendo mejoras significativas para problemas grandes. Además la versión híbrida MPI/OmpSs reduce substancialmente la complejidad del código y se ve menos afectada por el ancho de banda de la red y el ruido del sistema operativo que la versión MPI pura. Esta tesis también analiza y compara otros métodos actuales para solapar computación y comunicación colectiva, tales como usar comunicación punto a punto con hilos adicionales para la comunicación. La tesis resalta la importancia de entender las características de la computación que se ejecuta simultáneamente con la comunicación. Los resultados experimentales se han obtenido usando el benchmark sintético CCUBE ("Communication Computation Concurrent", Comunicación Computación Concurrente), desarrollado en esta tesis, además de HPL.
Cited by 0Open access
Liu Yin · 2014
With the recent development on heterogeneous hardware, heterogeneous parallel programming model has been widely used with the intension of simplifying programming and improving efficiency. This paper analyses latest achievements in heterogeneous parallel programming interfaces and runtime supporting systems, and solutions to new problems brought by heterogeneous architectures and various applications. In the end, some future trends in this area are discussed.
Cited by 0
Benoit Corda, K.H. Warren · 2003
The Parallel Fortran Preprocessor (PFP) is a programming model for multiple instruction multiple data (MIMD) parallel computers. It provides a simple paradigm consisting of data storage modifiers and parallel execution control statements. The model is lightweight and scalable in nature. The control constructs impose no implicit synchronizations, nor do they require off-processor memory references. The model is portable. It is implemented as a source-to-source translator which requires very little support from the back-end compiler. The implementation has an option to option to produce serial code which can then be compiled for serial execution.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Cited by 2
Françoise Baudé, Guy Vidal-Naquet · Lecture notes in computer science · 2005
Cited by 13
Eva Burrows, Magne Haveraaen · The Journal of Logic and Algebraic Programming · 2009
Cited by 6
Juan J. Gálvez, Karthik Senthil, Laxmikant V. Kalé · 2018
Parallel programming can be extremely challenging. Programming models have been proposed to simplify this task, but wide acceptance of these remains elusive for many reasons, including the demand for greater accessibility and productivity. In this paper, we introduce a parallel programming model and framework called CharmPy, based on the Python language. CharmPy builds on Charm++, and runs on top of its C++ runtime. It presents several unique features in the form of a simplified model and API, increased flexibility, and the ability to write everything in Python. CharmPy is a high-level model based on the paradigm of distributed migratable objects. It retains the benefits of the Charm++ runtime, including dynamic load balancing, asynchronous execution model with automatic overlap of communication and computation, high performance, and scalability from laptops to supercomputers. By being Python-based, CharmPy also benefits from modern language features, access to popular scientific computing and data science software, and interoperability with existing technologies like C, Fortran and OpenMP. To illustrate the simplicity of the model, we will show how to implement a distributed parallel map function based on the Master-Worker pattern using CharmPy, with support for asynchronous concurrent jobs. We also present performance results running stencil code and molecular dynamics mini-apps fully written in Python, on Blue Waters and Cori supercomputers. For stencil3d, we show performance similar to an equivalent MPI-based program, and significantly improved performance for imbalanced computations. Using Numba to JIT-compile the critical parts of the code, we show performance for both mini-apps similar to the equivalent C++ code.
Cited by 14
Bonetta, Daniele, Pautasso, Cesare · reroDoc Digital Library · 2015
Recent trends in programming models for server-side development have shown an increasing popularity of event-based single- threaded programming models based on the combination of dynamic languages such as JavaScript and event-based runtime systems for asynchronous I/O management such as Node.JS. Reasons for the success of such models are the simplicity of the single-threaded event-based programming model as well as the growing popularity of the Cloud as a deployment platform for Web applications. Unfortunately, the popularity of single-threaded models comes at the price of performance and scalability, as single-threaded event-based models present limitations when parallel processing is needed, and traditional approaches to concurrency such as threads and locks don't play well with event-based systems. This dissertation proposes a programming model and a runtime system to overcome such limitations by enabling single-threaded event-based applications with support for speculative parallel execution. The model, called Parallel Event Loop, has the goal of bringing parallel execution to the domain of single-threaded event-based programming without relaxing the main characteristics of the single-threaded model, and therefore providing developers with the impression of a safe, single-threaded, runtime. Rather than supporting only pure single-threaded programming, however, the parallel event loop can also be used to derive safe, high-level, parallel programming models characterized by a strong compatibility with single-threaded runtimes. We describe three distinct implementations of speculative runtimes enabling the parallel execution of event-based applications. The first implementation we describe is a pessimistic runtime system based on locks to implement speculative parallelization. The second and the third implementations are based on two distinct optimistic runtimes using software transactional memory. Each of the implementations supports the parallelization of applications written using an asynchronous single-threaded programming style, and each of them enables applications to benefit from parallel execution.
Cited by 0
Jameela Al‐Jaroodi, Nader Mohamed · 2004
This paper introduces an object-passing model for parallel and distributed application development. Object passing provides the object-oriented application developer with powerful yet simple methods to distribute and exchange data and logic (objects) among processes. The model extends message passing, while exploiting the advantages of the object-oriented paradigm. In addition, the model provides a portable framework for executing applications across multiple platforms, thus effectively exploiting available resources to gain more processing power. A number of advantages aspects adopting object passing are discussed, in addition to highlighting the differences between message passing, represented by MPI, and object passing. Another advantage is the model's suitability for heterogeneous systems. When implemented with a portable language like Java, it can support parallel and distributed applications spanning a collection of heterogeneous platforms. This form of execution will eventually allow for full utilization of available resources for any given application written using this model.
Cited by 0
I‐Chen Wu, H. T. Kung, Peter Steenkiste et al. · Defense Technical Information Center (DTIC) · 1993
Parallel programming requires task scheduling to optimize performance; this primarily involves balancing the load over the processors. In many cases, it is critical to perform task scheduling at runtime. For example, (1) in many parallel applications the task load cannot be accurately predicted a priori; (2) in a network-based multicomputer the computational power of each processor may not remain constant. In order to support dynamic task scheduling, the programmer, usually needs to design and implement a complex set of scheduling routines, e.g., routines for maintaining task lists and handling interprocessor communication for load balancing. Unfortunately, it is very difficult and time-consuming to write and debug all of these scheduling routines. This thesis proposes a new approach which can greatly reduce the effort of developing efficient dynamic task scheduling routines. In our new approach, we decompose task scheduling into two parts - the specification of scheduling policies and the implementation of supportive scheduling operations - and then hide the latter from the programmer. We call this approach multilist scheduling, because it is based on a uniform scheduling model involving the use of multiple scheduling lists.
Cited by 6