Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “data parallelism”

Data parallelism.

Kritpawit Soongswang (19469311), Chantana Chantrapornchai (19469314) · Figshare · 2024

<div><p>In this paper, we propose a method to reduce the model architecture searching time. We consider MobileNetV2 for 3D face recognition tasks as a case study and introducing the layer replication to enhance accuracy. For a given network, various layers can be replicated, and effective replication can yield better accuracy. Our proposed algorithm identifies the optimal layer replication configuration for the model. We considered two acceleration methods: distributed data-parallel training and concurrent model training. Our experiments demonstrate the effectiveness of the automatic model finding process for layer replication, using both distributed data-parallel and concurrent training under different conditions. The accuracy of our model improved by up to 6% compared to the previous work on 3D MobileNetV2, and by 8% compared to the vanilla MobileNetV2. Training models with distributed data-parallel across four GPUs reduced model training time by up to 75% compared to traditional training on a single GPU. Additionally, the automatic model finding process with concurrent training was 1,932 minutes faster than the distributed training approach in finding an optimal solution.</p></div>

Cited by 0Open access

GP-GPU: from data parallelism to stream parallelism

CECCONI, MARIA CHIARA · Electronic Theses and Dissertations Repository (University of Pisa) · 2019

L'obbiettivo della tesi è sperimentare il comportamento di una GPU, utilizzata per problemi non strettamente Data parallel, ampliando così l'ormai crescente range di utilizzi studiati per le General Purpose GPUs. In particolare, in questa tesi si è cercato di sperimentare, su processori grafici NVIDIA, il comportamento di uno Streaming parallel pattern, chiamato Farm. A tale scopo si è implementato un codice che facesse un utilizzo particolare delle risorse di una GPU: a partire da uno stream di dati in input (lato host), si creano molti task data parallel di dimensioni ridotte, da mandare in elaborazione sulla GPU. Utilizzando tanti CUDA Streams, quanti sono gli Streaming Multiprocessors (SM), ci si aspetta di osservare un occupazione completa della GPU. Soprattutto, si confronta il tempo di completamento del caso in cui non si usino CUDA Streams, con quello che si ottiene, invece, utilizzandoli. Quindi lo Speedup massimo atteso, tra le due suddette implementazioni, è minore o uguale al numero di SM di cui una GPU dispone. Studiando le performance su tre diversi tipi di kernel (altamente computazionale, moltiplicazione di matrici e image processing) si è osservato che, in effetti, si ottiene uno speedup vicino a quello atteso. Questo si verifica principalmente in applicazioni computation-bound, piuttosto che memory-bound.

Cited by 0Open access

Data and Task Parallelism

Narayan Ganesan, Roger D. Chamberlain, Jeremy Buhler et al. · 2012

Many biologically motivated problems are expressed as dynamic programming recurrences and are difficult to parallelize due to the intrinsic data dependencies in their algorithms. Therefore their solutions have been sped up using task level parallelism only. Emerging platforms such as GPUs are appealing parallel architectures for high-performance; at the same time they are a motivation to rethink the algorithms associated with these problems, to extract finergrained parallelism such as data parallelism. In this paper, we consider the hmmersearch program as a representative of these problems and we re-design its computational algorithm to extract data parallelism for a more efficient execution on emerging platforms, despite the fact that hmmersearch has data dependencies. Our approach outperforms other existing methods when searching a very large database of unsorted sequences on GPUs. 1.

Cited by 0

Parallelism for Big Data

2020

In 2004, Google published the seminal paper in which they described the process “MapReduce” that allows for a parallel processing model, to process huge amounts of data in parallel nodes. The Apache foundation succeeded to implement the MapReduce framework framework in an open-source solution “Hadoop.” However, Hadoop had limitations and implementation issues that pushed Apache to produce in 2012 “Spark.” Within the Hadoop project and the wider Hadoop ecosystem, Spark takes a special place with the resilient distributed dataset concept. Spark can be run stand alone or on a variety of platforms including Hadoop, Cassandra, and HBase. The chapter also illustrates how one can harvest the power of Spark via R.

Cited by 0

PROBLEMS WITH DATA PARALLELISM

Coretta Phillips, R. H. Perrott · Parallel Processing Letters · 2001

The gradual evolution of language features and approaches used for the programming of distributed memory machines underwent substantial advances in the 1990s. One of the most promising and widely praised approaches was based on data parallelism and resulted in High Performance Fortran. This paper reports on an experiment using that approach based on a commercial distributed memory machine, available compilers and simple test programs. The results are disappointing and not encouraging. The variety of components involved and the lack of detailed knowledge available for the compilers compound the difficulties of obtaining results and doing comparisons. The results show great variation and question the premise that communication is the decisive factor in performance determination. The results are also a contribution towards the difficult tasks of predicating performance on a distributed memory computer.

Cited by 3

Abstractions for Adaptive Data Parallelism

P. Roe · 1996

ions for Adaptive Data Parallelism P. Roe School of Computing Science Queensland University of Technology Brisbane, Queensland, 4001 p.roe@qut.edu.au Abstract This paper describes how a class of data parallel programs (SPMD) may be expressed using reusable adaptive abstractions. The abstractions support adaptive use of a network of workstations for parallel computing. Although data parallelism is the paradigm considered the programs are expressed using message passing. The main contribution of this paper is to demonstrate how adaptive parallelism may be realised using abstractions, their costs and benefits. 1 Introduction In the aggregate networks of workstations represent a huge unused computing resource. Workstations are, on average, largely unused: in any one minute during peak time, 60% of all machines are on average idle [3]. This paper investigates how networks of idle workstations can be adaptively used for parallel computation. The utilisation of a dynamic set of workstati...

Cited by 0

Exploiting data-parallelism in gpus

Frank Mueller, Yongpeng Zhang · 2012

Mainstream microprocessor design no longer delivers performance boosts by increasing the processor clock frequency due to power and thermal constraints. Nonetheless, advances in semiconductor fabrication still allow the transistor density to increase at the rate of Moore’s law. This has resulted in the proliferation of many-core parallel architectures and accelerators, among which GPUs (graphics processing unit) quickly established themselves as suitable for applications that exploit fine-grained data-parallelism. GPU clusters are starting to make inroads into the HPC (high performance computing) domain as well, due to much better power per Flop (floating point operation) performance than general-purpose processors such as CPUs. Even though it is easier to program GPUs than ever, efficiently taking advantage of GPU resources requires unique techniques that are not found elsewhere. The traditional function level task-parallelism can hardly provide enough optimization opportunities for such parallel architectures. Instead, it is crucial to extract data-parallelism and map it to the massive threading execution model advocated by GPUs. This dissertation consists of multiple efforts to build programming models above existing models (CUDA) for single GPUs as well as GPU clusters. We start from manually implementing a flocking-based document clustering algorithm on GPU clusters. With this first-hand experience to write code directly above CUDA and MPI (message passing interface), we make several key observations: (1) Unified memory interface greatly enhances programmability, especially in GPU cluster environment, (2) explicit expression of data parallelism at language level facilitates the mapping of algorithms to massively parallel architectures and (3) auto-tuning is necessary to achieve competitive performance as the parallel architecture becomes more complex. Based on these observations, we propose several programming models and compiler approaches to achieve portability and programmability while retaining as much performance as possible. • We propose GStream, a general-purpose, scalable data streaming framework on GPUs. We project powerful, yet concise language abstractions onto GPUs to fully exploit their inherent massive data-parallelism. • We take a domain specific language approach to provide an efficient implementation of 3D iterative stencil computations on GPUs with auto-tuning capabilities. • We propose CuNesl, a compiler framework to translate and optimize a nested data-parallel language called NESL into parallel CUDA programs for SIMT architectures. By converting recursive calls into while loops, we ensure that the hierarchical execution model in GPUs can be exploited on the “flattened” code. • Finally, we design HiDP, a hierarchical data-parallel language suitable for hierarchical features of microprocessor architectures. We then develop a source-to-source compiler that converts HiDP into CUDA C++ source code with tuning capability. It greatly improves coding productivity while still keeping up with the performance of hand-coded CUDA code. The methods above cover a wide range of techniques for GPGPU computing and represent the current technology trend to exploit data parallelism in state-of-the-art GPUs.

Cited by 0

Integrating Task and Data Parallelism

Berna L. Massingill · CaltechAUTHORS (California Institute of Technology) · 1993

Many models of concurrency and concurrent programming have been proposed; most can be categorized as either task-parallel (based on functional decomposition) or data-parallel (based on data decomposition). Task-parallel models are most effective for expressing irregular computations; data-parallel models are most effective for expressing regular computations. Some computations, however, exhibit both regular and irregular aspects. For such computations, a better programming model is one that integrates task and data parallelism. This report describes one model of integrating task and data parallelism, some problem classes for which it is effective, and a prototype implementation.

Cited by 0Open access

Work-efficient nested data-parallelism

Daniel W. Palmer, Jan F. Prins, Stephen J. Westfold · 2002

An apply-to-all construct is the key mechanism for expressing data-parallelism, but data-parallel programming languages like HPF and C* significantly restrict which operations can appear in the construct. Allowing arbitrary operations substantially simplifies the expression of irregular and nested data-parallel computations. The technique of flattening nested parallelism introduced by Blelloch, compiles data-parallel programs with unrestricted apply-to-all constructs into vector operations, and has achieved notable success, particularly with irregular data-parallel programs. However, these programs must be carefully constructed so that flattening them does not lead to suboptimal work complexity due to unnecessary replication in index operations. We present new flattening transformations that generate programs with correct work complexity. Because these transformations may introduce concurrent reads in parallel indexing, we developed a randomized indexing that reduces concurrent reads while maintaining work-efficiency. Experimental results show that the new rules and implementations significantly reduce memory usage and improve performance.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Cited by 20

Mixed Task and Data Parallelism

Olivier Beaumont, Arnaud Legrand, Yves Robert · HAL (Le Centre pour la Communication Scientifique Directe) · 2002

International audience

Cited by 1