Allen D. Malony A, Sameer S. Shende A, Alan Morris A · 2008
Parallel scientific applications are designed based on structural, logical, and numerical models of computation and correctness. When studying the performance of these applications, especially on large-scale parallel systems, there is a strong preference among developers to view performance information with respect to their “mental model ” of the application, formed from the model semantics used in the program. If the developer can relate performance data measured during execution to what they know about the application, more effective program optimization may be achieved. This paper considers the concept of “phases ” and its support in parallel performance measurement and analysis as a means to bridge the gap between high-level application semantics and low-level performance data. In particular, this problem is studied in the context of parallel performance profiling. The implementation of phase-based parallel profiling in the TAU parallel performance system is described and demonstrated for the NAS parallel benchmarks and MFIX application. 1.
Cited by 0
Fan Zhang · 2010
With the development of parallel technology and application, the performance analysis and visualization of parallel computing is one of the most important parts. This paper presents a parallel computing performance visualization system which is developed to visualize and analyze the speedup and efficiency of parallel computing. We visualize and analyze the results of magnetotelluric parallel forward modeling program based on MPI which used finite element method. It succeeds in collecting the data of program execution time and communication time, visualizing and analyzing the speedup and efficiency.
Cited by 4
Tao Wen, Yuanjuan Yang, Ling Zhu et al. · 2023
Reduction and critical are two methods to synchronize access to shared variables in parallel regions in OpenMP, which have a certain impact on the performance of many parallel computations. Therefore, the parallel speedup ratio of reduction and critical has a certain research value. In this paper, three common mathematical problems such as solving calculus equation, determinant expansion and matrix multiplication are taken as examples to write corresponding parallel programs. The parallel parts of the program are executed by using reduction and critical instructions respectively, and the speedup ratios of the above two instructions in program execution are emphatically tested under different calculation scales and different thread numbers. The results show that the performance of reduction instruction is better than that of critical instruction when a shared variable is required to be specified. When the task scale is small, critical instruction can improve the performance of the program, but with the increase of the task scale, critical instruction will become the bottleneck of the program and affect the scalability of the program. In OpenMP programming, programmers need to choose different instructions flexibly according to the nature and requirements of tasks in order to obtain better performance and efficiency.
Cited by 0
Isaac Dooley, Chee Wai Lee, Laxmikant V. Kalé · 2009
Traditional performance analysis techniques are performed after a parallel program has completed. In this paper, we describe an online method for continuously monitoring the performance of a parallel program, specifically the fraction of the time spent in various activities as the program executes. Our implementation of both a visualization client and the parallel performance framework that gathers utilization data are described. The data gathering uses a scalable and asynchronous reduction with an appropriate lossless compressed data format. The overheads in the initial system are low, even when run on thousands of processors. The data gathering occurs in an out-of-band communication mechanism, interleaving itself transparently with the execution of the parallel application by leveraging a message-driven runtime system.
Cited by 1
Michael T. Heath, Jennifer E. Finger · 2003
ParaGraph is a graphical display tool for visualizing the behavior and performance of parallel programs that use MPI (Message-Passing Interface). The visual animation of a parallel program is based on execution trace information gathered during an actual run of the program on a message-passing parallel computer system. The resulting trace data are replayed pictorially to provide a dynamic depiction of the behavior of the parallel program, as well as graphical summaries of its overall performance. The same performance data can be viewed from many different visual perspectives to gain insights that might be missed by any single view. We describe the visualization tool, discuss the motivation and philosophy behind its design, and provide details on how to use it.
Cited by 0
Yeh‐Ching Chung, Sanjay Ranka · 2002
The authors propose a parallel programming tool, PPT, to aid programmers in finding the best program partitioning and scheduling and to automatically generate the parallel code for the single program multiple data (SPMD) model on a distributed memory multiprocessor. In a distributed memory multiprocessor, the program partitioning and scheduling play an important role in the performance of a parallel program. The goal of PPT is to aid programmers in designing a parallel program that can be run on a distributed memory multiprocessor efficiently with balanced load and low communication cost. PPT has six components: a program partitioning and directed acyclic graph (DAG) generator, a DAG analyzer, a scheduler, a communication analyzer, a code generator, and a performance evaluator. Each of these components is described.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Cited by 0
Matthew B. Dwyer, Matthias Martin · 1999
. Once the sole province of super-computing centers, parallel computing systems are now becoming common-place. Parallel systems are regularly used to boost the performance of, for example, multi-user systems and web-servers. In general, exploiting parallelism to improve the performance of a single application remains a significant challenge. One approach to meeting this challenge is to leverage existing developer experience that has been codified in concurrent design patterns and associated programming frameworks. In this paper, we report on our experiences using a parallel programming framework to implement a parallel version of the FLAVERS program analysis system. We describe the effort required to modify an existing sequential implementation of FLAVERS, the improved performance of the resulting parallel FLAVERS, and the prospects for generalizing this approach to other program analyses. Keywords : program analysis, flow analysis, shared-memory parallelism, programming ...
Cited by 0
William Evan Speight · Rice Digital Scholarship Archive (Rice University) · 2009
Performance debugging is the process of isolating and correcting performance problems in an otherwise correct parallel program. Problems not immediately visible to the parallel programmer often lead to poor application performance. This thesis describes the design, implementation, and use of ParaView, a tool to locate performance inefficiencies in programs written for shared-memory multiprocessors. ParaView supplies an intuitive, graphical interface based upon the X-windows system. ParaView aids parallel applications programmers in uncovering performance bugs relating to poor cache performance, load balancing, false sharing, and inefficient synchronization. Eleven parallel programs have been analyzed using ParaView, and performance limitations in five of these were improved. Reductions in overall execution times range from 25% to 86% for sixteen processor simulations. Our experience demonstrates that ParaView facilitates parallel program performance debugging by reducing the amount of time required to uncover and correct performance problems relating to poor data partitioning, false sharing, contention for shared data constructs, and unnecessary synchronization.
Cited by 1Open access
Celso L. Mendes, Jhy-chun Wang, Daniel A. Reed · 1995
With data parallel languages like High Performance Fortran (HPF), an application developer &apos;s mental model of program execution can differ markedly from the code that actually executes on a particular parallel system. In such situations, understanding the reasons for poor program performance requires a time consuming cycle of performance experimentation and code tuning where one seeks to understand the relation between data parallel source and the performance of compiler-synthesized SPMD code. Because performance is the primary motivation for parallel computing, it is critical to understand the potential scalability of a code as a function of the number of processors (P ) and problem size (N ). It is equally important that this understanding be related to the high-level data parallel source code, not just to the SPMD code. In this paper, we propose a performance scalability model for data parallel code fragments based on symbolic expressions of N and P . In this approach, the data para...
Cited by 0
T. Fahringer, P. Blaha, A. Hössinger et al. · 1999
Several large real-world applications have been developed for distributed and parallel architectures. We examine two different program development approaches: First, the usage of a high-level programming paradigm which reduces the time to create a parallel program dramatically but sometimes at the cost of a reduced performance. A source-to-source compiler, has been employed to automatically compile programs – written in a high-level programming paradigm – into message passing codes. Second, manual program development by using a low-level programming paradigm – such as message passing – enables the programmer to fully exploit a given architecture at the cost of a time-consuming and error-prone effort. Performance tools play a central role to support the performance-oriented development of applications for distributed and parallel architectures. SCALA – a portable instrumentation, measurement, and post-execution performance analysis system for distributed and parallel programs – has been used to analyse and to guide the application development by selectively instrumenting and measuring the code versions, by comparing performance information of several program executions, by computing a variety of important performance metrics, by detecting performance bottlenecks, and by relating performance information back to the input program. We show several experiments of SCALA when applied to real-world applications. These experiments are conducted for a NEC Cenju-4 distributed memory machine and a cluster of heterogeneous workstations and networks.
Cited by 0
Oscar Naím, Tony Hey · 1996
Representation of Parallel Program Performance Oscar Na&apos;im, Tony Hey, and Ed Zaluska Abstract Performance is a critical issue in current massively parallel processors. However, delivery of adequate performance is not automatic and performance evaluation tools are required in order to help the programmer to understand the behaviour of a parallel program. In recent years, a wide variety of tools have been developed for this purpose including tools for monitoring and evaluating performance and visualization tools. However, these tools do not provide an abstract representation of performance. Massively parallel processors can generate a huge amount of performance data and sophisticated methods for representing and displaying this data (e.g. visual and aural) are required. Performance views are not scalable in general and do not represent an abstraction of the performance data. The Do-Loop-Surface display is proposed as an abstract representation of the performance of a partic...
Cited by 0
Kunal Agrawal, Jeremy T. Fineman, Kefu Lu et al. · 2014
Although concurrent data structures are commonly used in practice on shared-memory machines, even the most efficient concurrent structures often lack performance theorems guaranteeing linear speedup for the enclosing parallel program. Moreover, efficient concurrent data structures are difficult to design. In contrast, parallel batched data structures do provide provable performance guarantees, since processing a batch in parallel is easier than dealing with the arbitrary asynchrony of concurrent accesses. They can limit programmability, however, since restructuring a parallel program to use batched data structure instead of concurrent data structure can often be difficult or even infeasible.
Cited by 26
Seng Wai Loke · 1994
One problem of parallel computing is that parallel computers vary greatly in architecture so that a program written to run efficiently on a particular architecture, when porting to a different architecture, would often need to be changed and adapted substantially in order to run with reasonable performance on the target architecture. Porting with performance is, hence, labour-intensive and costly. A method of parallel programming using the Bird-Meertens Formalism where programs are formulated as compositions of (mainly) higher order functions on some data type in the data parallel functional style has been proposed as a solution. The library of (mainly) higher-order functions in which all communication and parallelism in a program is embedded could (it is argued) be implemented efficiently on different parallel architectures. This gives the advantage of portability between different architectures with reasonable and predictable performance without change in program source. ...
Cited by 0
Wentong Cai, Hung Khoon Tan · 1994
In this paper, we investigate the performance of parallel discreteevent simulation using PVM, and the feasibility of using PVM for object-oriented parallel program development. We find that PVM can be very easily adapted by C programmers for parallel programming on a network of workstations, but the performance of the program tends to depend on the workload of each workstation. In our study, both deadlock avoidance and time-warp approaches are used for time synchronization in parallel simulation, and the performance analysis are conducted on simulating Omega networks and public telephone networks.The design and implementation of our simulation program are described. The results obtained are also presented. Keywords: parallel discrete-event simulation, deadlock avoidance, time-warp, PVM, Omega network &amp; telephone network simluation, object-oriented programming. 1. Introduction Discrete-event simulation demands high computational requirements as the problem size increases [13]. With th...
Cited by 0
Deokhwan Kim, Saša Misailovíc, Martin Rinard · DSpace@MIT (Massachusetts Institute of Technology) · 2010
Traditional parallelizing compilers are designed to generate parallel programs that produce identical outputs as the original sequential program. The difficulty of performing the program analysis required to satisfy this goal and the restricted space of possible target parallel programs have both posed significant obstacles to the development of effective parallelizing compilers. The QuickStep compiler is instead designed to generate parallel programs that satisfy statistical accuracy guarantees. The freedom to generate parallel programs whose output may differ (within statistical accuracy bounds) from the output of the sequential program enables a dramatic simplification of the compiler and a significant expansion in the range of parallel programs that it can legally generate. QuickStep exploits this flexibility to take a fundamentally different approach from traditional parallelizing compilers. It applies a collection of transformations (loop parallelization, loop scheduling, synchronization introduction, and replication introduction) to generate a search space of parallel versions of the original sequential program. It then searches this space (prioritizing the parallelization of the most time-consuming loops in the application) to find a final parallelization that exhibits good parallel performance and satisfies the statistical accuracy guarantee. At each step in the search it performs a sequence of trial runs on representative inputs to examine the performance, accuracy, and memory accessing characteristics of the current generated parallel program. An analysis of these characteristics guides the steps the compiler takes as it explores the search space of parallel programs. Results from our benchmark set of applications show that QuickStep can automatically generate parallel programs with good performance and statistically accurate outputs. For two of the applications, the parallelization introduces noise into the output, but the noise remains within acceptable statistical bounds. The simplicity of the compilation strategy and the performance and statistical acceptability of the generated parallel programs demonstrate the advantages of the QuickStep approach.
Cited by 7Open access
Min Li · 2010
In this paper parallel program was performed for the FDTD electromagnetic algorithm on a multi-core CPU computer with CUDA device. In the case of two-dimension FDTD method, computation times were measured and compared to investigate the efficiency of parallel FDTD application. The results show that CUDA parallel program can improve the computing performance efficiently which can be used for three-dimension FDTD parallel applications.
Cited by 2
Paul Roe · 2007
. This paper describes how the evaluation of parallel programs may be simulated by transforming a parallel program into a sequential one which generates simulation information in addition to its standard results. This technique allows the programmer to investigate the performance of key programs and algorithms in an abstract way but without having to solve complicated recurrence equations. The main advantage of this technique is its flexibility. It is shown how by using this technique, several different kinds of information concerning a parallel program&apos;s behaviour may be generated, including: average parallelism measurements and task profiles. A functional language is used as the candidate parallel language, and Quicksort is used as the example program. 1 Introduction The raison d&apos;etre for parallel programming is increased performance. Unfortunately parallel programs have complex behaviours, which makes designing and reasoning about them difficult. Thus reasoning about parallel progr...
Cited by 0
Ko‐Yang Wang · Purdue e-Pubs (Purdue University System) · 1990
In this paper. we introduce a performance prediction framework for parallel compilers based on the refined combilled characterization model (CCM). The CCM model characterizes the perfonnance of a user program on a target machine by combining a set of characteristic functions called perfonnance factors. Each perfonnance factor represents a particular program aspect that relates program pat~ terns to features of a target architecture and is quantified by an evaluation function. The evaluation functions arc usually inexpensive to compute. so they can be evaluated repeatedly during the parallelism optimization process. The performance prediction framework is highly flexible and can be adjusted with a knOWledge base to fit different needs at different stages of parallel compiling and to accommodate different classes of parallel architectures. The performance prediction model is utilized in an intelligent parallel compiler to guide the program optimization process. Its versatility allows the compiler to offer an adjustable optimization degree and can optimize code for a wide range of target machines.
Cited by 8Open access
Xiong, Lang-lang, Wang, Xi-min, Liu, Song et al. · arXiv (Cornell University) · 2017
The finite-difference time-domain (FDTD) method has been commonly utilized to simulate the electromagnetic (EM) waves propagation in the plasma media. However, the FDTD method may bring about extra run-time on concerning computationally large and complicated EM problems. Fortunately, the FDTD method is easy to parallelize. Besides, GPU has been widely used for parallel computing due to its unique SPMD (Single Program Multiple Data) architecture. In this paper, we represent the parallel Runge-Kutta exponential time differencing scheme FDTD (RKETD) method for the unmagnetized plasma implemented on GPU. The detailed flowchart of parallel RKETD-FDTD method is described. The accuracy and acceleration performance of the proposed parallel RKETD-FDTD method implemented on GPU are substantiated by calculating the reflection and transmission coefficients for one-dimensional unmagnetized plasma slab. The results indicate that the numerical precision of the parallel RKETD-FDTD scheme is consistent with that of the code implemented on CPU. The computation efficiency is greatly improved compared with merely CPU-based serial RKETD-FDTD method. Moreover, the comparisons of the performance of CUDA-based GPU parallel program, OpenMP (Open Multi-Processing)-based CPU parallel program, and single-CPU serial program on the same host computer are done. Compared with the serial program, both parallel programs get good results, while GPU-based parallel program gains better result.
Cited by 0Open access
Purdue E-pubs, Ko-yang Wang, Ko-yang Wang et al. · 1990
In this paper. we introduce a performance prediction framework for parallel compilers based on the refined combilled characterization model (CCM). The CCM model characterizes the perfonnance of a user program on a target machine by combining a set of characteristic functions called perfonnance factors. Each perfonnance factor represents a particular program aspect that relates program pat~ terns to features of a target architecture and is quantified by an evaluation function. The evaluation functions arc usually inexpensive to compute. so they can be evaluated repeatedly during the parallelism optimization process. The performance prediction framework is highly flexible and can be adjusted with a knOWledge base to fit different needs at different stages of parallel compiling and to accommodate different classes of parallel architectures. The performance prediction model is utilized in an intelligent parallel compiler to guide the program optimization pro-cess. Its versatility allows the compiler to offer an adjustable optimization degree and can optimize code for a wide range of target machines.
Cited by 0