Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 4,062 papers for “Task parallelism”

Optimal Utilization of Computing Resources with Data Parallelism and Task Parallelism in building AI models

Vamshi Malthummeda · International Journal of Innovative Research in Engineering & Multidisciplinary Physical Sciences · 2025

This article focuses on techniques for the most effective utilization of computing infrastructure for training and tuning the AI models in a distributed fashion used across various industries for making informed decisions. The objective is to investigate the potential benefits of data parallelism and task parallelism to optimize computing infrastructure which is in the form spark clusters on databricks. In addition the paper aims to illustrate how data parallelism and task parallelism can be achieved by setting up Ray cluster on top of databricks spark cluster to train and tune the Time series forecasting models to forecast sales of various products at stores in retail industry. The paper also discusses the significance of data parallelism and task parallelism in driving down the AI model building compute cost and time. The paper aims to provide the Machine Learning Engineers, Organization IT sponsors and all other stake holders actionable insights towards adoption of Ray cluster on top of databricks spark cluster. It presents detailed analysis of Time series data forecasting using Ray + Spark clusters combination facilitated by databricks. The paper aims to aid the organizations in enhancing the efficiency of computing resources, reducing the operational costs and speeding up the process of forecast model building using the solution presented.

Cited by 0Open access

Modeling the Benefits of Mixed Data and Task Parallelism

Soumen Chakrabarti James, James Demmel, Katherine Yelick · 1995

Mixed task and data parallelism exists naturally in many applications, but utilizing it may require sophisticated scheduling algorithms and software support. Recently, significant research effort has been applied to exploiting mixed parallelism in both theory and systems communities. In this paper, we ask how much mixed parallelism will improve performance in practice, and how architectural evolution impacts these estimates. First, we build and validate a performance model for a class of mixed task and data parallel problems based on machine and problem parameters. Second, we use this model to estimate the gains from mixed parallelism for some scientific applications on current machines. This quantifies our intuition that mixed parallelism is best when either communication is slow or the number of processors is large. Third, we show that, for balanced divide and conquer trees, a simple one-time switch between data and task parallelism gets most of the benefit of general mixed paralleli...

Cited by 0

GOMP Profiler: A Profiler for OpenMP Task Level Parallelism

Joseph M. Arul, Gwang-Jie Hwang, Han-Yao Ko · 2013

The current trend in microprocessor is to go multi -core, be it in desktops, note-books or hand-held systems. Fortunately, many major software and hardware vendors specifically defined a parallel programming API model called OpenMP. Recently, OpenMP version 3.0 included a new feature, i.e., task level parallelism, for parallelizing unstructured programs. This research focuses on using a particular instrumented library, together with the OpenMP run-time lib rary, for a specific GNU Co mp iler Co llection (GCC) and constructs a visualization tool for presenting the performance during task level parallelism. In order to verify the capability of the GNU OpenMP (GOMP) Profiler, Barcelona OpenMP Task Suits (BOTS) were used and analysed. Four different metrics were used in order to eliminate the unnecessary overheads and understand the behavior of BOTS. Thus, by using this profiler, a p rogrammer can understand the code better and rearrange the code in order to avoid the extra co mputations involved during the parallel execution. Thus, task level parallelis m can be improved greatly.

Cited by 1

A Unified Scheduler for Recursive and Task Dataflow Parallelism

Hans Vandierendonck, George Tzenakis, Dimitrios S. Nikolopoulos · Research Portal (Queen's University Belfast) · 2011

Task dataflow languages simplify the specification of parallel programs by dynamically detecting and enforcing dependencies between tasks. These languages are, however, often restricted to a single level of parallelism. This language design is reflected in the runtime system, where a master thread explicitly generates a task graph and worker threads execute ready tasks and wake-up their dependents. Such an approach is incompatible with state-of-the-art schedulers such as the Cilk scheduler, that minimize the creation of idle tasks (work-first principle) and place all task creation and scheduling off the critical path. This paper proposes an extension to the Cilk scheduler in order to reconcile task dependencies with the work-first principle. We discuss the impact of task dependencies on the properties of the Cilk scheduler. Furthermore, we propose a low-overhead ticket-based technique for dependency tracking and enforcement at the object level. Our scheduler also supports renaming of objects in order to increase task-level parallelism. Renaming is implemented using versioned objects, a new type of hyper object. Experimental evaluation shows that the unified scheduler is as efficient as the Cilk scheduler when tasks have no dependencies. Moreover, the unified scheduler is more efficient than SMPSS, a particular implementation of a task dataflow language.

Cited by 35Open access

Novel List Scheduling Strategies for Data Parallelism Task Graphs

Yang Liu, Lin Meng, Ittetsu Taniguchi et al. · International Journal of Networking and Computing · 2014

This paper studies task scheduling algorithms which schedule a set of tasks on multiple cores so that the total scheduling length is minimized. Most of the algorithms developed in the past assume that a task is executed on a single core. Unlike the previous algorithms, the algorithms studied in this paper allow a task to be executed on multiple cores. This paper proposes six algorithms. All of the six algorithms are based on list scheduling, but the strategy for priority assignment is different. In our experiments, the six algorithms as well as an integer linear programming method are evaluated.

Cited by 15Open access

A Coordination Layer for Exploiting Task Parallelism with HPF

Salvatore Orlando, Raffaele Perego · 1998

. This paper introduces COLT HPF , a run--time support for exploiting task parallelism within HPF programs, which can be employed by a compiler of a high-level coordination language to structure a set of data-parallel HPF tasks according to popular paradigms of task-parallelism. We use COLT HPF to program a computer vision application and report the results obtained by running the application on an SGI/Cray T3E. 1 Introduction Although HPF-1 [8] permits programmers to express data-parallel computations in a portable, high-level way, it is widely accepted that many important parallel applications cannot be efficiently implemented following a pure data-parallel paradigm. The promising possibility of exploiting a mixture of task and data parallelism, where data parallelism is restricted within HPF tasks and task parallelism is achieved by their concurrent execution, has recently received much attention [6, 5]. Depending on the applications, HPF tasks can be organized according to patter...

Cited by 0

Supporting intra-task parallelism in real-time multiprocessor systems

José Fonseca · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · 2012

Os sistemas de tempo real modernos geram, cada vez mais, cargas computacionais pesadas e dinâmicas, começando-se a tornar pouco expectável que sejam implementados em sistemas uniprocessador. Na verdade, a mudança de sistemas com um único processador para sistemas multi- processador pode ser vista, tanto no domínio geral, como no de sistemas embebidos, como uma forma eficiente, em termos energéticos, de melhorar a performance das aplicações. Simultaneamente, a proliferação das plataformas multi-processador transformaram a programação paralela num tópico de elevado interesse, levando o paralelismo dinâmico a ganhar rapidamente popularidade como um modelo de programação. A ideia, por detrás deste modelo, é encorajar os programadores a exporem todas as oportunidades de paralelismo através da simples indicação de potenciais regiões paralelas dentro das aplicações. Todas estas anotações são encaradas pelo sistema unicamente como sugestões, podendo estas serem ignoradas e substituídas, por construtores sequenciais equivalentes, pela própria linguagem. Assim, o modo como a computação é na realidade subdividida, e mapeada nos vários processadores, é da responsabilidade do compilador e do sistema computacional subjacente. Ao retirar este fardo do programador, a complexidade da programação é consideravelmente reduzida, o que normalmente se traduz num aumento de produtividade. Todavia, se o mecanismo de escalonamento subjacente não for simples e rápido, de modo a manter o overhead geral em níveis reduzidos, os benefícios da geração de um paralelismo com uma granularidade tão fina serão meramente hipotéticos. Nesta perspetiva de escalonamento, os algoritmos que empregam uma política de workstealing são cada vez mais populares, com uma eficiência comprovada em termos de tempo, espaço e necessidades de comunicação. Contudo, estes algoritmos não contemplam restrições temporais, nem outra qualquer forma de atribuição de prioridades às tarefas, o que impossibilita que sejam diretamente aplicados a sistemas de tempo real. Além disso, são tradicionalmente implementados no runtime da linguagem, criando assim um sistema de escalonamento com dois níveis, onde a previsibilidade, essencial a um sistema de tempo real, não pode ser assegurada. Nesta tese, é descrita a forma como a abordagem de work-stealing pode ser resenhada para cumprir os requisitos de tempo real, mantendo, ao mesmo tempo, os seus princípios fundamentais que tão bons resultados têm demonstrado. Muito resumidamente, a única fila de gestão de processos convencional (deque) é substituída por uma fila de deques, ordenada de forma crescente por prioridade das tarefas. De seguida, aplicamos por cima o conhecido algoritmo de escalonamento dinâmico G-EDF, misturamos as regras de ambos, e assim nasce a nossa proposta: o algoritmo de escalonamento RTWS. Tirando partido da modularidade oferecida pelo escalonador do Linux, o RTWS é adicionado como uma nova classe de escalonamento, de forma a avaliar na prática se o algoritmo proposto é viável, ou seja, se garante a eficiência e escalonabilidade desejadas. Modificar o núcleo do Linux é uma tarefa complicada, devido à complexidade das suas funções internas e às fortes interdependências entre os vários subsistemas. Não obstante, um dos objetivos desta tese era ter a certeza que o RTWS é mais do que um conceito interessante. Assim, uma parte significativa deste documento é dedicada à discussão sobre a implementação do RTWS e à exposição de situações problemáticas, muitas delas não consideradas em teoria, como é o caso do desfasamento entre vários mecanismo de sincronização. Os resultados experimentais mostram que o RTWS, em comparação com outro trabalho prático de escalonamento dinâmico de tarefas com restrições temporais, reduz significativamente o overhead de escalonamento através de um controlo de migrações, e mudanças de contexto, eficiente e escalável (pelo menos até 8 CPUs), ao mesmo tempo que alcança um bom balanceamento dinâmico da carga do sistema, até mesmo de uma forma não custosa. Contudo, durante a avaliação realizada foi detetada uma falha na implementação do RTWS, pela forma como facilmente desiste de roubar trabalho, o que origina períodos de inatividade, no CPU em questão, quando a utilização geral do sistema é baixa. Embora o trabalho realizado se tenha focado em manter o custo de escalonamento baixo e em alcançar boa localidade dos dados, a escalonabilidade do sistema nunca foi negligenciada. Na verdade, o algoritmo de escalonamento proposto provou ser bastante robusto, não falhando qualquer meta temporal nas experiências realizadas. Portanto, podemos afirmar que alguma inversão de prioridades, causada pela sub-política de roubo BAS, não compromete os objetivos de escalonabilidade, e até ajuda a reduzir a contenção nas estruturas de dados. Mesmo assim, o RTWS também suporta uma sub-política de roubo determinística: PAS. A avaliação experimental, porém, não ajudou a ter uma noção clara do impacto de uma e de outra. No entanto, de uma maneira geral, podemos concluir que o RTWS é uma solução promissora para um escalonamento eficiente de tarefas paralelas com restrições temporais.

Cited by 0Open access

Dynamic estimation of task level parallelism with operating system

Luong Dinh Hung, Shotaro Sakai · 2005

The amount of task level parallelism (TLP) in runtime workload is useful information to determine the efficient usage of multiprocessors. This paper presents mechanisms to dynamically estimate the amount of TLP in runtime workloads. Modifications are added to the operating system (OS) to collect information about processor utilization, task activities, from which TLP can be calculated. By effectively utilizing the time stamp counter (TSC) hardware, the task activities can be monitored at fine time resolution, resulting in capability of estimation of TLP at fine granularity. We implement the mechanisms on a recent version of Linux OS. Evaluation results indicate that the mechanisms can estimate TLP accurately for various kinds of workloads with small overheads.

Cited by 0

Task parallelism for ray tracing on a gpu cluster

Ünlü, Çağlar · OpenMETU (Middle East Technical University) · 2019

Ray tracing is a computationally complex global illumination algorithm that is used for producing realistic images. In addition to parallel implementations on commodity PC clusters, recently, Graphics Processing Units (GPU) have also been used to accelerate ray tracing. In this thesis, ray tracing is accelerated on a GPU cluster where the viewing plane is divided into unit tiles. Slave processes work on these tiles in a task parallel manner which are dynamically assigned to them. To decrease the number of ray-triangle intersection tests, Bounding Volume Hierarchies (BVH) are used. It is shown that almost linear speedup can be achieved. On the other hand, it is observed that API and network overheads are obstacles for scalability.

Cited by 0Open access

Conceptual diagram of space discretization and task-based parallelism.

T. J. Sego (9904909), James P. Sluka (3124269), Herbert M. Sauro (188028) et al. · Figshare · 2023

<p>Space is discretized into subdomains called “cells” (shown here for N subdomains), and particles (listed as “PX”) are stored in memory by which cell contains them. For each simulation step, a set of threads, called “runners” (shown here for M runners) perform a pre-defined set of tasks. The “Sort” task builds ordered lists of particle indices according to proximity to neighboring cells for efficient pruning of inter-cell interactions between particle pairs outside of the cutoff distance. The “Force Self” task calculates interactions between particles of the same cell. The “Force Pair” task calculates interactions between particles of different cells using results from the Sort task. Task scheduling enforces task dependency. After all forces are calculated, particle positions are updated and particles that move into a neighboring cell are appropriately moved in memory, as demonstrated for particles P247 (from cell N to cell 1) and P233 (from cell 1 to cell 2).</p>

Cited by 0Open access

Puncalc: task-based parallelism and speculative reevaluation in spreadsheets

Alexander Asp Bock, Florian Biermann · The Journal of Supercomputing · 2019

Spreadsheets are commonly declarative, first-order functional programs and are used as organizational tools, for end-user development and for educational purposes. Spreadsheet end users are usually domain experts who use spreadsheets as their main computational model, but are seldom trained IT professionals who can leverage today’s abundant multicore processors for spreadsheet computation. In this paper, we present an algorithm for automatic, parallel evaluation of spreadsheets targeting shared-memory multicore architectures, which lets end users transparently make use of their multicore processors. We evaluate our algorithm on a set of synthetic and real-world spreadsheets and obtain up to 16 times speedup on 48 cores.

Cited by 4Open access

Task Parallelism to Optimize Performance of Environmental Modeling Software

Althea Denlinger · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2021

Climate modeling is an integral part of environmental research, from studying rare phenomena to predicting future climate trends. The need for more accurate models is only growing, but as climate modeling capabilities advance, existing workflows require optimization to recoup performance. A solution comes in the form of task parallelism, a novel programming capability that provides an opportunity for optimization at execution time by allowing tasks to be executed in parallel, reducing runtime significantly. Using Parsl, an intuitive and scalable parallel scripting library for Python, we implement task parallelism within support software to aid in the continuous advancement of climate modeling technology.

Cited by 0Open access

A Deep Dive into Task-Based Parallelism in Python

William Ruys, Hochan Lee, Bozhi You et al. · 2024

Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. First, we perform an experimental study to examine the impact of Python's Global Interpreter Lock (GIL), and potential speedups under a GIL-less PEP703 future, to guide runtime design. Using our optimized runtime, we explore scheduling tasks with constraints that require resources across multiple, potentially diverse, devices through the introduction of new programming abstractions and runtime mechanisms. We extend an existing Python tasking library, Parla, to augment its performance and add support for such multi-device tasks. Our experimental analysis, using tasks graphs from synthetic and real applications, shows at least a 3 ×(and up to 6 ×) performance improvement over its predecessor in scenarios with high GIL contention. When scheduling multi-GPU tasks, we observe an 8x reduction in per-task launching overhead compared to a multi-process system.

Cited by 3