Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.
About 128 papers for “embarrassingly parallel algorithms”
Hesham Ali, Salah Attiya, Ibrahim Elhenawy · arXiv (Cornell University) · 2017
In this paper we develop parallel cluster sampling algorithms and show that a\nmulti-chain version is embarrassingly parallel and can be used efficiently for\nmedical image retrieval among other applications.\n
Cited by 0Open access
Willie Neiswanger, Chong Wang, Eric P. Xing · arXiv (Cornell University) · 2015
We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating with other machines. This type of "embarrassingly parallel" procedure has recently been developed for MCMC inference algorithms; however, in many cases it is not possible to directly extend this procedure to VI methods without requiring certain restrictive exponential family conditions on the form of the model. Furthermore, most existing (nonparallel) VI methods are restricted to use on conditionally conjugate models, which limits their applicability. To combat these issues, we make use of the recently proposed nonparametric VI to facilitate an embarrassingly parallel VI procedure that can be applied to a wider scope of models, including to nonconjugate models. We derive our embarrassingly parallel VI algorithm, analyze our method theoretically, and demonstrate our method empirically on a few nonconjugate models.
Cited by 9Open access
Satish Narayana Srirama, Oleg Batrashev, Pelle Jakovits et al. · SHILAP Revista de lepidopterología · 2011
Cloud computing, with its promise of virtually infinite resources, seems to suit well in solving resource greedy scientific computing problems. To study the effects of moving parallel scientific applications onto the cloud, we deployed several benchmark applications like matrix–vector operations and NAS parallel benchmarks, and DOUG (Domain decomposition On Unstructured Grids) on the cloud. DOUG is an open source software package for parallel iterative solution of very large sparse systems of linear equations. The detailed analysis of DOUG on the cloud showed that parallel applications benefit a lot and scale reasonable on the cloud. We could also observe the limitations of the cloud and its comparison with cluster in terms of performance. However, for efficiently running the scientific applications on the cloud infrastructure, the applications must be reduced to frameworks that can successfully exploit the cloud resources, like the MapReduce framework. Several iterative and embarrassingly parallel algorithms are reduced to the MapReduce model and their performance is measured and analyzed. The analysis showed that Hadoop MapReduce has significant problems with iterative methods, while it suits well for embarrassingly parallel algorithms. Scientific computing often uses iterative methods to solve large problems. Thus, for scientific computing on the cloud, this paper raises the necessity for better frameworks or optimizations for MapReduce.
Cited by 0Open access
Norman Matloff · 2013
The growth in the use of computationally intensive statistical procedures, especially with Big Data, has necessitated the usage of parallel computation on diverse platforms such as multicore, GPU, clusters and clouds. How-ever, slowdown due to interprocess communication costs typically limits such methods to “embarrassingly parallel ” (EP) algorithms, especially on non-shared memory platforms. This paper develops a broadly-applicable method for converting many non-EP algorithms into statistically equivalent EP ones. The method is shown to yield excellent levels of speedup for a variety of statistical computations. It also overcomes certain problems of memory limitations. 1
Cited by 0
Ilja Kromonov, Pelle Jakovits, Satish Narayana Srirama · 2013
The importance of fault tolerance for the parallel computing field is ever increasing, as the mean time between failures is predicted to decrease significantly for future highly parallel systems. The current trend of using commodity hardware to reduce the cost of clusters forces users to ensure that their applications are fault tolerant. When it comes to embarrassingly parallel data-intensive algorithms, MapReduce has gone a long way in simplifying the creation of such applications. However, this does not apply to iterative communication-intensive algorithms common in the scientific computing domain. In this work we propose a new programming model inspired by Bulk Synchronous Parallel (BSP) for creating new a fault tolerant distributed computing framework. We strive to retain the advantages that MapReduce provides, yet efficiently support a larger assortment of algorithms, such as the aforementioned iterative ones.
Cited by 1
Flagg, Raymond F, III · DigitalCommons (California Polytechnic State University) · 2015
GPGPUs offer significant computational power for programmers to leverage. This computational power is especially useful when utilized for accelerating scientific models. This thesis analyzes the utilization of GPGPU programming to accelerate scientific computing models. First the construction of hardware for visualization and computation of scientific models is discussed. Several factors in the construction of the machines focus on the performance impacts related to scientific modeling. Image processing is an embarrassingly parallel problem well suited for GPGPU acceleration. An image processing library was developed to show the processes of recognizing embarrassingly parallel problems and serves as an excellent example of converting from a serial CPU implementation to a GPU accelerated implementation. Genetic algorithms are biologically inspired heuristic search algorithms based on natural selection. The Tetris genetic algorithm with A* pathfinding discusses memory bound limitations that can prevent direct algorithm conversions from the CPU to the GPU. An analysis of an existing landscape evolution model, CHILD, for GPU acceleration explores that even when a model shows promise for GPU acceleration, the underlying data structures can have a significant impact upon that ability to move to a GPU implementation. CHILD also offers an example of creating tighter MATLAB integration between existing models. Lastly, a parallel spatial sorting algorithm is discussed as a possible replacement for current spatial sorting algorithms implemented in models such as smoothed particle hydrodynamics.
Cited by 0Open access
Pan Wei · 2011
Since analyzing large-scale graph is usually difficult to be implemented on a single machine,how to design efficient parallel large-scale graph algorithms is receiving more and more attention.Constrained by embarrassingly parallel assumption,parallel graph algorithms are not easy to express in MapReduce.Inspired by Bulk Synchronous Parallel model,we propose a message-enhanced version of Hadoop MapReduce that breaks its key assumption.Enhanced implementation is compatible with original Hadoop MapReduce,existing Hadoop MapReduce programs can run directly on this platform without modification,and uses message passing mechanisms to facilitate interactive data communication between supersteps of tasks.It also provides a highly flexible self-defined message passing interface and two adaptive message passing mechanisms to support efficient implementation of graph algorithms with data transition and iterative computation.The experimental results on the real Stanford large network dataset collection demonstrate the superiority of enhanced version over original Hadoop MapReduce on PageRank algorithm.
Cited by 4
2019
This chapter explains some of the issues that arise when one tries to use multiple processors to solve a single problem. It describes different models of parallel processing and explains some algorithms and techniques that he/she can use to solve parallelizable problems more quickly. Currently, distributed computing is the most common model for most people. The chapter discusses general issues that face distributed algorithms: debugging and identifying embarrassingly parallel problems. It also explains how one can use mergesort on multiple processors. One may not need to use some of the more esoteric algorithms, such as the zero-time sort on a systolic array or the solution to the dining philosophers problem, but all of these algorithms highlight some of the problems that can arise in distributed systems. Those problems include such issues as race conditions, deadlock, livelock, consistency, and synchronization.
Cited by 5
Frank Krämer, Andreas Leha, Tim Beißbarth · 2011
Newly developed classifiers, estimators, and other algorithms are often tested in simulations to assess power, control of alpha-level, accuracy and related quality criteria. Depending on the required number of simulation runs and the complexity of algorithms, especially the estimation of parameters and the drawing of random numbers under certain distributions, these simulations can run for several hours or even days. Such simulations are embarrassingly parallel and, therefore, benefit massively from parallelization. Not everyone has access to clusters or grids, though, but highly parallel graphics cards suitable for general purpose computing are installed in many computers. While there is a distinct number of maintained R-packages available (Eddelbuettel2011) that are able to interface with GPUs and allow the user to speed up computations, integrating
Cited by 0
Raghavendra D Prabhu · 2006
Abstract — Artificial Neural networks have been part of an attempt to emulate the learning curve of the human nervous system. However the vital difference of, nervous system being highly parallel and computer processor units remaining largely sequential persists. Here an attempt is made to bridge that gap with the help of Graphics Processing Units (GPUs) which are designed to be highly parallel. In particular Back propagation networks are considered which use supervised learning. Back-propagation algorithms, with no data dependencies are embarrassingly parallel and hence only a totally parallel system can exploit it fully. However, it has also been observed that GPUs underperform when either significant overhead in calculations is incurred or algorithm is not sufficiently parallel. Index Terms — feed forward neural networks, Graphics processing units, Back-propagation networks, data-parallelism
Cited by 0
Semester Credits, Instructor Dr. Amit Jain · 2014
computation. Fundamental design patterns used in parallel algorithms: embarrassingly par-allel, partitioning, divide and conquer, software pipelining, synchronous computations and load balancing. Implementation of parallel programs using MPI, GPUs and Map-Reduce on parallel clusters. PREREQ: CS 253 and CS 321 or PERM/INST. Prerequisites The student should be familiar with the fundamental data structures and algorithms used in sequential computing. The course on Data Structures (CS 321) should provide the requisite background. Also the student should have prior experience with C programming, which the course CS 253 provides. Goals Parallel computing covers a wide area in computer science and an introductory course cannot span all the topics. The emphasis in this course is to teach students how to “think in parallel.” Fundamental ideas that are not likely to change with time are stressed. By taking this course, the student will be able to: 1 • describe parallel models of computation and parallel architectures, • explain the fundamental concepts in parallel computing, • solve problems faster by using multiple processors, whether “in a box ” or spread across a cluster, • design parallel programs for problems, • convert sequential programs to parallel programs, • and write parallel programs for at least one parallel system.
Cited by 0
Al Globus, James M. Crawford, Jason Lohn et al. · NASA STI Repository (National Aeronautics and Space Administration) · 2002
We describe work in progress concerning multi-instrument, multi-satellite scheduling. Most, although not all, Earth observing instruments currently in orbit are unique. In the relatively near future, however, we expect to see fleets of Earth observing spacecraft, many carrying nearly identical instruments. This presents a substantially new scheduling challenge. Inspired by successful commercial applications of evolutionary algorithms in scheduling domains, this paper presents work in progress regarding the use of evolutionary algorithms to solve a set of Earth observing related model problems. Both the model problems and the software are described. Since the larger problems will require substantial computation and evolutionary algorithms are embarrassingly parallel, we discuss our parallelization techniques using dedicated and cycle-scavenged workstations.
Cited by 57Open access
Heinz Stockinger, Marco Pagni, Lorenzo Cerutti et al. · 2006
Bioinformatics algorithms such as sequence alignment methods based on profile-HMM (Hidden Markov Model) are popular but CPU-intensive. If large amounts of data are processed, a single computer often runs for many hours or even days. High performance infrastructures such as clusters or computational Grids provide the techniques to speed up the process by distributing the workload to remote nodes, running parts of the work load in parallel. Biologists often do not have access to such hardware systems. Therefore, we propose a new system using a modern Grid approach to optimise an embarrassingly parallel problem. We achieve speed ups by at least two orders of magnitude given that we can use a powerful, world-wide distributed Grid infrastructure. For large-scale problems our method can outperform algorithms designed for mid-size clusters even considering additional latencies imposed by Grid infrastructures.
Cited by 19
Ethan T. Almquist, Aaron W. Langham, Stephen J. Bruno et al. · IEEE Access · 2026
Traditional microgrid modeling approaches based on differential equations are computationally expensive. These physics-based simulations from first principles sometimes provide more fidelity than is required for analysis and design tasks. Behavioral simulation and modeling move the computational burden from numerical solving algorithms to time-domain bookkeeping. This paper presents techniques to make the behavioral simulation of a microgrid ‘‘embarrassingly parallel,’’ allowing modern multicore CPUs to throw their entire weight at the problem. Techniques are developed to remove state dependencies which inhibit parallelization. Benchmarks and a case study on real shipboard microgrid data serve as validation.
Cited by 0Open access
Society for Industrial and Applied Mathematics eBooks · 2001
The primary motivation for using parallel computers is to obtain faster turnaround time for the computations. In addition, parallel machines offer access to more memory than single processors. Achieving high efficiency on parallel machines is closely related to the choice of algorithms and to optimal data distribution among the processing elements. As interprocessor communication is the costliest component of any parallel algorithm, algorithms well suited for parallelism are those that can be divided into relatively independent subproblems that can be solved without much data exchange. Such algorithms are generically called “coarse-grained.” Algorithms of this type are not known for all computational problems. Another essential ingredient for parallel efficiency is the optimization of the serial code, already extensively discussed. In fact, parallel optimization should be done only after the serial program is optimal. This strategy is likely to lead to best parallel performance and avoids artificially inflated speedup numbers. In this chapter we will first discuss metrics for parallel performance and then offer some guidelines on how to optimize parallel programs. 8.1 Ideal and observed speedup One measure of parallel efficiency is the “parallel speedup,” which tells how many times faster a program runs on a parallel machine compared to a single processor of the same type. If N is the number of processors (or processing elements), the ideal speedup Si is Si =N. The ideal speedup is a theoretical limit, not attainable except for a very small subset of real problems called “embarrassingly parallel.” Such a program typically requires just the distribution and collection of some data at the beginning and at the end of the run.
Cited by 0
Gabriele D’Angelo, M Mandrioli, Moreno Marzolla · Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) · 2013
Identifying intersections among a set of d-dimensional rectangular regions (d-rectangles) is a common problem in many simulation and modeling applications. Since algorithms for computing intersections over a large number of regions can be computationally demanding, an obvious solution is to take advantage of the multiprocessing capabilities of modern multicore processors. Unfortunately, many solutions employed for the Data Distribution Management service of the High Level Architecture are either inefficient, or can only partially be parallelized. We propose the Interval Tree Matching (ITM) algorithm for computing intersections among d-rectangles. ITM is based on a simple Interval Tree data structure, and exhibits an embarrassingly parallel structure. The sequential implementation of ITM is competitive with sort-based matching; moreover, the parallel implementation provides good speedup on multicore processors.
Cited by 0
Vijayalakshmi Saravanan, Sai Karthik Navuluru, Khaled Z. Ibrahim · 2024
Scientific data management is crucial for researchers dealing with immense datasets on high-performance computing systems, especially at the Peta scale. Yet, traditional file systems impose constraints on parallel data output, necessitating novel approaches to enhance parallelism and refine resource allocation. As the number of nodes grows, robust data organization becomes critical to mitigate failure risks, while efficient parallel processing is vital for accelerating computations and optimizing resource utilization. This study introduces an analytical model for quantitatively assessing the performance of large-scale datasets using parallelization techniques, encompassing both massively and embarrassingly parallel methods. While these algorithms provide approximate performance bounds, practical applications often rely on more efficient approaches. Through experimental analysis of real-world scientific datasets such as Open Catalyst and NWCHEM, and exploration of optimization techniques, this study offers valuable insights for optimizing parallelization strategies, propelling advancements in parallel computing techniques within scientific research and data analysis realms.
Cited by 0
Willie Neiswanger, Chong Wang, Eric Xing · 2016
Abstract. Communication costs, resulting from synchronization requirements during learning, can greatly slow down many parallel machine learning algorithms. In this paper, we present a parallel Markov chain Monte Carlo (MCMC) algorithm in which subsets of data are processed independently, with very little communication. First, we arbitrarily partition data onto multiple machines. Then, on each machine, any classical MCMC method (e.g., Gibbs sampling) may be used to draw samples from a posterior distribution given the data subset. Finally, the samples from each machine are combined to form samples from the full posterior. This embarrassingly parallel algorithm allows each machine to act independently on a subset of the data (without communication) until the final combination stage. We prove that our algorithm generates asymptotically exact samples and empirically demonstrate its ability to parallelize burn-in and sampling in several models. 1.
Cited by 0
Willie Neiswanger, Chong Wang, Eric P. Xing · 2014
Communication costs, resulting from synchro-nization requirements during learning, can greatly slow down many parallel machine learning algorithms. In this paper, we present a parallel Markov chain Monte Carlo (MCMC) algorithm in which subsets of data are pro-cessed independently, with very little com-munication. First, we arbitrarily partition data onto multiple machines. Then, on each machine, any classical MCMC method (e.g., Gibbs sampling) may be used to draw samples from a posterior distribution given the data subset. Finally, the samples from each ma-chine are combined to form samples from the full posterior. This embarrassingly parallel algorithm allows each machine to act inde-pendently on a subset of the data (without communication) until the final combination stage. We prove that our algorithm generates asymptotically exact samples and empirically demonstrate its ability to parallelize burn-in and sampling in several models. 1
Cited by 0
Linlin Xu, Giray Ökten · arXiv (Cornell University) · 2014
GPU computing has become popular in computational finance and many financial institutions are moving their CPU based applications to the GPU platform. Since most Monte Carlo algorithms are embarrassingly parallel, they benefit greatly from parallel implementations, and consequently Monte Carlo has become a focal point in GPU computing. GPU speed-up examples reported in the literature often involve Monte Carlo algorithms, and there are software tools commercially available that help migrate Monte Carlo financial pricing models to GPU. We present a survey of Monte Carlo and randomized quasi-Monte Carlo methods, and discuss existing (quasi) Monte Carlo sequences in GPU libraries. We discuss specific features of GPU architecture relevant for developing efficient (quasi) Monte Carlo methods. We introduce a recent randomized quasi-Monte Carlo method, and compare it with some of the existing implementations on GPU, when they are used in pricing caplets in the LIBOR market model and mortgage backed securities.
Cited by 0Open access