Yongwei Zhao, Yunji Chen, Zhiwei Xu · Intelligent Computing · 2022
As machine learning (ML) becomes the prominent technology for many emerging problems, dedicated ML computers are being developed at a variety of scales, from clouds to edge devices. However, the heterogeneous, parallel, and multilayer characteristics of conventional ML computers concentrate the cost of development on the software stack, namely, ML frameworks, compute libraries, and compilers, which limits the productivity of new ML computers. Fractal von Neumann architecture (FvNA) is proposed to address the programming productivity issue for ML computers. FvNA is scale-invariant to program, thus making the development of a family of scaled ML computers as easy as a single node. In this study, we generalize FvNA to the field of general-purpose parallel computing. We model FvNA as an abstract parallel computer, referred to as the fractal parallel machine (FPM), to demonstrate several representative general-purpose tasks that are efficiently programmable. FPM limits the entropy of programming by applying constraints on the control pattern of the parallel computing systems. However, FPM is still general-purpose and cost-optimal. We settle some preliminary results showing that FPM is as powerful as many fundamental parallel computing models such as BSP and alternating Turing machine. Therefore, FvNA is also generally applicable to various fields other than ML.
Cited by 1Open access
D.J. Pease · 1992
Cited by 13
George Almási, Allan Gottlieb · CERN Document Server (European Organization for Nuclear Research) · 1989
Part 1 Foundations: overview - overview and scope of this book, definition and driving forces, questions raised, emerging answers, previous attempts why success now?, conclusions and future directions sample applications - scientific and engineering applications, database systems, artificial intelligence systems, summary technological constraints and opportunities - processor and network technology, memory technology, storage technology computational models and selected algorithms - computational models an operational view, computational models - an analytical view, selected parallel algorithms. Part 2 Parallel software: languages and programming environments - review of the major serial languages, parallel imperative languages and extensions, declarative languages, the programmer's view compilers, other translators - serial compiler essentials, parallelizing compiler essentials, summary and perspective operating systems - operating systems for serial machines, controlling concurrency, classifying operating systems for parallel computers, history of parallel operating systems. Part 3 Parallel architectures: interconnection networks - static connection topologies, dynamic connection topologies SIMD parallel architectures - evolution from von Neumann machines, vector processors, pipelined SIMD vector processors, parallel SIMD designs MIMD parallel architectures - stepping up to MIMD, private memory (message-passing), MIMD designs, shared memory MIMD designs hybrid parallel architectures - VLIW architectures, MSIMD tree machines, MSIMD reconfigurable designs.
Cited by 528
Nikolay Mirenkov · Parallel Computing Technologies · 1991
Cited by 7
Hans‐Christian Hege, Renate Knecht · Parallel Computing · 1992
Cited by 13
Fox, Geoffrey C, Williams, Roy D, Messina, Guiseppe C · CERN Bulletin · 2014
Cited by 15
Ramón Díaz-uriarte, Daniel Rico · 2013
Cited by 0
Marc Snir · 1993
No abstract available.
Cited by 81Open access
Hana ˇ Sevčíková, A. J. Rossini · 2009
Two of the biggest practical problems facing parallel statistical computing are random number generation and fault tolerance. This paper describes some recent work from our research group which addresses these issues. This research has resulted in the packages rlecuyer, snowFT, and in contributions to the snow package.
Cited by 0
R. D. Williams, Guiseppe C. Messina, Geoffrey Fox et al. · 1994
1 Introduction 2 Technical Backdrop 3. A Methodology for Computation 4 Synchronous Applications I 5 Express and CrOS 6 Synchronous Applications II 7 Independent Parallelism 8 Full Matrix Algorithms 9 Loosely Synchronous Problems 10 DIME Programming Environment 11 Load Balancing and Optimization 12 Irregular LS Problems 13 Data Parallel C and Fortran 14 Asynchronous Applications 15 High-Level Asynchronous Software 16 The Zipcode Message-Passing System 17 MOVIE 18 Simulation and Analysis 20 Computational Science
Cited by 257
2021
Cited by 0
Vinod Kumar Khanna · 2025
Flynn’s classification of computer organization for fast computers according to the number of simultaneously manipulated instructions/data items, consists of categories named as single stream of instruction with single stream of data (SISD), single stream of instruction with multiple streams of data (SIMD), multiple streams of instructions with single stream of data (MISD) and multiple streams of instructions with multiple streams of data (MIMD). Distinctive problems facing processing units for artificial intelligence (AI) are outlined. AI computing lays a heavy burden on the processor of a computer, which can be appreciably relieved by adopting a parallel computing architecture. Parallel programming patterns include the master–slave pattern, divide-and-conquer pattern, pipelining and speculative multithreading. Among the types of parallelism, mention may be made of bit-level parallelism; data parallelism including vector processing; task parallelism including the pipelining scheme; and instruction-level parallelism (ILP) including the superscalar processor architecture, out-of-order execution, speculative execution, and multiple issue processors. Advantages and disadvantages of ILP are enumerated. Parallelism in software engineering is done by application checkpointing, automatic parallelization and through parallel programming languages. Several types of parallel processors are discussed, notably the pipelined, vector, SIMD array and attached array processors. Multicore and multiprocessor processors are demarcated. Asynchronous operation of an array processor is described.
Cited by 0
H.E. Plumblee · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 1990
The architecture and performance of massively parallel computers and their applicability to aerospace problems are discussed with particular reference to results obtained with a parallel machine using 16,384 processors. The discussion focuses on work conducted in the field of computational fluid dynamics and electromagnetics. The cost/performance of a massively parallel computer is evaluated.
Cited by 2
Roman Trobec, Gregor Kosec · SpringerBriefs in computer science · 2015
Cited by 23
Jackson, R.O. · OpenGrey (Institut de l'Information Scientifique et Technique) · 1999
Available from British Library Document Supply Centre-DSC:DXN044891 / BLDSC - British Library Document Supply Centre
Cited by 0
Pacura, Dávid · Brno University of Technology Digital Library (Brno University of Technology) · 2013
Cieľom tejto bakalárskej práce je poukázanie na moţnosti a demonštrácia paralelizácie výpočtov pomocou grafických procesorov. V práci sú uvedené popisy dostupných vývojových nástrojov, a na ich základe je jeden zvolený na implementáciu šifrovacieho MD5 algoritmu a neurónovej siete na rozpoznávanie čísel. Výkon algoritmov je následne porovnaný so svojím paralelným ekvivalentom pre beţné procesory. V závere sú popísané problémy pri vývoji a spôsoby, ktorými je moţné sa im vyhnúť
Cited by 0
Collective · CERN Document Server (European Organization for Nuclear Research) · 1989
Cited by 0
Victor Malyshkin · Lecture notes in computer science · 2023
Cited by 0Open access
Victor Malyshkin · Lecture notes in computer science · 1999
Cited by 252
Nicolas Szczepanski · HAL (Le Centre pour la Communication Scientifique Directe) · 2017
Cited by 0Open access