Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 746 papers for “hardware prefetching”

Hardware Based Prefetching Methods

Angeline Ebenezer · 1998

This paper is intended to address the hardware based technique and will address both instruction and data prefetching. One major advantage of the

Cited by 0

Best-offset hardware prefetching

Pierre Michaud · HAL (Le Centre pour la Communication Scientifique Directe) · 2016

Hardware prefetching is an important feature of modern high-performance processors. When the application working set is too large to fit in on-chip caches, disabling hardware pre-fetchers may result in severe performance reduction. A new prefetcher was recently introduced, the Sandbox prefetcher, that tries to find dynamically the best prefetch offset using the sandbox method. The Sandbox prefetcher uses simple hardware and was shown to be quite effective. However, the sandbox method does not take into account prefetch timeliness. We propose an offset prefetcher with a new method for selecting the prefetch offset that takes into account prefetch timeliness. We show that our Best-Offset prefetcher outperforms the Sandbox prefetcher on the SPEC CPU2006 benchmarks, with equally simple hardware.

Cited by 185Open access

Last-level collective hardware prefetching

Georgios Michelogiannakis · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2023

A last-level collective hardware prefetcher (LLCHP) is described. The LLCHP is to detect a first off-chip memory access request by a first processor core of a plurality of processor cores. The LLCHP is further to determine, based on the first off-chip memory access request, that first data associated with the first off-chip memory access request is associated with second data of a second processor core of the plurality of processor cores. The LLCHP is further to prefetch the first data and the second data based on the determination.

Cited by 0Open access

Hardware Prefetching Techniques for Cache Memories . . .

R. Cucchiara, et al. · 2000

The workload of multimedia applications has a strong impact on cache memory performance, since the locality of memory references embedded in multimedia programs differs from that of traditional programs. In many cases, standard cache memory organization achieves poorer performance when used for multimedia. A widely explored approach to improve cache performance is hardware prefetching that allows the pre-loading of data in the cache before they are referenced. However, existing hardware prefetching approaches partially miss the potential performance improvement, since they are not tailored to multimedia locality. In this paper we propose novel effective approaches to hardware prefetching to be used in image processing programs for multimedia. Experimental results are reported for a suite of multimedia image processing programs including convolutions with kernels, MPEG-2 decoding, and edge chain coding.

Cited by 0

Merging Similar Patterns for Hardware Prefetching

Shizhi Jiang, Qiusong Yang, Yiwei Ci · 2022

One critical challenge of designing an efficient prefetcher is to strike a balance between performance and hardware overhead. Some state-of-the-art prefetchers achieve very high performance at the price of a very large storage requirement, which makes them not amenable to hardware implementations in commercial processors. We argue that merging memory access patterns can be a feasible solution to reducing storage overhead while obtaining high performance, although no existing prefetchers, to the best of our knowledge, have succeeded in doing so because of the difficulty of designing an effective merging strategy. After analysis of a large number of patterns, we find that the address offset of the first access in a certain memory region is a good feature for clustering highly similar patterns. Based on this observation, we propose a novel hardware data prefetcher, named Pattern Merging Prefetcher (PMP), which achieves high performance at a low cost. The storage requirement for storing patterns is largely reduced and, at the same time, the prefetch accuracy is guaranteed by merging similar patterns in the training process. In the prefetching process, a strategy based on access frequencies of prefetch candidates is applied to accurately extract prefetch targets from merged patterns. According to the experimental results on a wide range of various workloads, PMP outperforms the enhanced Bingo by 2.6% with 30× lesser storage overhead and Pythia by 8.2% with 6× lesser storage overhead.

Cited by 20

Hardware-driven prefetching for pointer data references

Chi‐Hung Chi, Chin-Ming Cheung · 1998

Article Hardware-driven prefetching for pointer data references Share on Authors: Chi-Hung Chi Dept. of Information Systems and Computer Science, National University of Singapore, 10 Lower Kent Ridge Crescent, Singapore 119260 Dept. of Information Systems and Computer Science, National University of Singapore, 10 Lower Kent Ridge Crescent, Singapore 119260View Profile , Chin-Ming Cheung Dept. of Information Systems and Computer Science, National University of Singapore, 10 Lower Kent Ridge Crescent, Singapore 119260 Dept. of Information Systems and Computer Science, National University of Singapore, 10 Lower Kent Ridge Crescent, Singapore 119260View Profile Authors Info & Claims ICS '98: Proceedings of the 12th international conference on SupercomputingJuly 1998 Pages 377–384https://doi.org/10.1145/277830.277924Online:13 July 1998Publication History 7citation322DownloadsMetricsTotal Citations7Total Downloads322Last 12 Months4Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Cited by 11

Advanced hardware prefetching in virtual memory systems

Georgios Vavouliotis · 2023

(English) Despite groundbreaking technological innovations, the disparity between processor and memory speeds (known as Memory Wall) is still a major performance obstacle for modern systems. Hardware prefetching is a latency-tolerance technique that has proven successful at shrinking this bottleneck. Nearly all real-world µarchitectural designs employ various prefetchers. Consequently, hardware prefetching attracts a lot of research attention. Virtual memory has been vital for the success of computing due to its programmability and security benefits. However, virtual memory does not come for free since each memory access requires a translation from the virtual to the physical address space that incurs high latency and energy overheads. To alleviate these overheads, a hardware cache, named Translation Lookaside Buffer(TLB), is typically employed to store the most recently used translations. However, TLBs are limited in capacity, thus there are not adequate for assuring high performance. Processor vendors address the need for fast address translation by providing dedicated support for virtual memory (e.g., multi-level TLBs, multiple page sizes). Despite the existence of such support, the advent of applications with large data and code footprints aggravates the pressure placed on the virtual memory subsystem, resulting in frequent page walks that deteriorate system's performance. This dissertation argues that hardware prefetching can attenuate the Memory Wall bottleneck in virtual memory systems. To support our claim, we design and propose fully-legacy preserving TLB prefetching schemes and exploit address translation metadata that are available at the µarchitecture to improve the effectiveness of the prefetchers operating in the physical address space. To reduce the overheads of frequent TLB misses due to data accesses, we propose a solution that consists of the Sampling-Based Free TLB Prefetching (SBFP) scheme and the Agile TLB Prefetcher(ATP). SBFP exploits the locality in the last-level of the page table to enhance the performance of TLB prefetching. ATP combines three prefetch engines while disabling TLB prefetching during phases that does not provide benefits. Across different benchmark suites, we show that ATP combined with SBFP improves performance over the best performing prior TLB prefetcher while reducing the page walk references to the memory hierarchy. Next, we argue that instruction address translation is an emerging bottleneck in servers. To support our claim, we characterize the TLB behavior of server workloads and provide evidence that instruction address translation is a bottleneck in servers. To attenuate this bottleneck, we propose Morrigan, the first instruction TLB prefetcher. Morrigan combines a sequential prefetcher with an ensemble of hardware Markov prefetchers that build variable length Markov chains out of the instruction TLB miss stream while using a new frequency-based replacement policy. Across a set of industrial server workloads, Morrigan provides great performance gains while eliminating the majority of the demand page walks for instruction accesses. Our last contribution improves the efficacy of cache prefetchers operating in the physical address space by exploiting modern support for large pages. We propose the Page-size Propagation Module(PPM), a µarchitectural scheme that transmits the page size information to the lower-level cache prefetchers and enables safe prefetching beyond 4KB physical page boundaries. We further design a module comprised of two prefetchers that both exploit PPM but drive prefetching decisions assuming different page sizes. Our experiments reveal that the proposed page size exploitation techniques provide great performance enhancements on various state-of-the-art cache prefetchers. ¿he proposals of this dissertation are fully legacy-preserving, do not call for disruptive changes, do not require any OS involvement, and constitute practical solutions to real-world bottlenecks. (Español) La diferencia de velocidad entre los procesadores y las memorias (conocido como Memory Wall) es uno de los mayores cuellos de botella en los sistemas actuales. Los prefetchers hardware son una técnica para tolerar la latencia de memoria que ha demostrado ser efectiva para aliviar este cuello de botella. Prácticamente todas las microarquitecturas actuales incluyen varios prefetchers. La memoria virtual ha sido vital para el éxito de los sistemas de computación debido a los beneficios que ofrece en términos de programabilidad y seguridad. Sin embargo, la memoria virtual tiene un coste alto en latencia y en energía, ya que cada acceso a memoria requiere una traducción del espacio de direcciones físico al virtual. Para aliviar estos costes, una caché hardware llamada Translation Lookahead Buffer (TLB) guarda las traducciones más recientes. Uno de los problemas de las TLBs es que tienen una capacidad limitada, lo que hace que no puedan garantizar un alto rendimiento. Los procesadores comerciales se aseguran de que la traducción de direcciones sea rápida al incluir soporte hardware dedicado a la memoria virtual (como TLBs multinivel o varios tamaños de página). Incluso con este soporte hardware, la creciente popularidad de aplicaciones con grandes cantidades de datos e instrucciones ha aumentado la presión sobre el subsistema de memoria virtual, resultando en accesos frecuentes a la tabla de páginas que deterioran el rendimiento del sistema. Esta tesis pretende atenuar el cuello de botella del Memory Wall utilizando hardware prefetchers en sistemas de memoria virtual. Para reducir las penalizaciones de los fallos frecuentes de TLB para accesos de datos, proponemos una solución llamada Sampling Based Free TLB Prefetching (SBFP) combinada con un Agile TLB Prefetcher (ATP). SBFP se beneficia de la localidad en el último nivel de la tabla de páginas para mejorar el prefetching en TLBs. ATP combina tres motores de prefetching y deshabilita el prefetching de TLB en fases donde no aporta ningún beneficio. Para varios juegos de prueba, demostramos que la combinación de ATP con SBFP mejora el rendimiento respecto al mejor prefetcher de TLB propuesto anteriormente, a la vez que reduce la cantidad de accesos a la tabla de páginas. También identificamos que la traducción de direcciones en instrucciones es un cuello de botella emergente en servidores, y lo demostramos caracterizando el comportamiento de las TLBs en estos sistemas. Para aliviar este cuello de botella proponemos Morrigan, el primer prefetcher de instrucciones para TLBs. Morrigan combina un prefetcher secuencial con un conjunto de Markov prefetchers que construyen cadenas de Markov de diferente longitud de la secuencia de fallos de instrucciones en la TLB, y también utiliza una nueva política de reemplazo basada en la frecuencia. En tareas de servidores industriales, Morrigan consigue amplios beneficios en rendimiento y elimina la mayoría de accesos a la tabla de páginas para accesos a instrucciones. La última contribución mejora la eficacia de los prefetchers de cachés que operan en el espacio de direcciones físico utilizando el soporte hardware para páginas grandes. Para ello proponemos el Page-size Propagation Module (PPM), una técnica microarchitectural que transmite el tamaño de página a los prefetchers de las cachés de último nivel para poder hacer prefetching de forma segura sobrepasando los límites de las páginas físicas de 4KB. También diseñamos otro módulo que consiste en dos prefetchers conectados al PPM y que controla las decisiones de los prefetchers asumiendo diferentes tamaños de páginas. Nuestros experimentos muestran que las técnicas propuestas para sacar beneficio del tamaño de las páginas en varios prefetchers de cachés de último nivel propuestas en el estado del arte consiguen grandes mejoras de rendimiento.

Cited by 0Open access

Cooperative Prefetching: Compiler and Hardware Support for Effective Instruction Prefetching in Modern Processors

Chi-Keung Luk Todd, Todd C. Mowry · 1998

Instruction cache miss latency is becoming an increasingly importantperformance bottleneck, especially for commercial applications. Although instruction prefetching is an attractive technique for tolerating this latency, we find that existing prefetching schemes are insufficient for modern superscalar processors since they fail to issue prefetches early enough (particularly for non-sequential accesses). To overcome these limitations, we propose a new instruction prefetching technique whereby the hardware and software cooperate to hide the latency as follows. The hardware performs aggressive sequential prefetching combined with a novel prefetch filtering mechanism to allow it to get far ahead without polluting the cache. To hide the latency of non-sequential accesses, we propose and implement a novel compiler algorithm which automatically inserts instructionprefetch instructions into the executable to prefetch the targets of control transfers far enough in advance. Our experimental resu...

Cited by 0

PARE: a power-aware hardware data prefetching engine

Yao Guo, M.B. Naser, C.A. Moritz · 2005

Aggressive hardware prefetching often significantly increases energy consumption in the memory system. Experiments show that a major fraction of prefetching energy degradation is due to the hardware history table related energy costs. In this paper, the authors presented PARE, a power-aware prefetching engine that uses a newly designed indexed hardware history table. Compared to the conventional single table design, the new prefetching table consumes 7-11X less power per access. With the help of compiler-based location-set analysis, it is shown that the proposed PARE design improves energy consumption by as much as 40% in the data memory systems in 70nm processor designs.

Cited by 1

Hardware-Only Stream Prefetching and Dynamic Access Ordering

2008

Memory system bottlenecks limit performance for many applications, and computations with strided access patterns are among the hardest hit. The streams used in such applications have extremely poor cache behavior. These access patterns have the advantage of being predictable, though, and this can be exploited to improve the efficiency of the memory subsystem in two ways: memory latencies can be masked by prefetching stream data, and the latencies can be reduced by reordering stream accesses to exploit parallelism and locality within the DRAMs. Many researchers have studied hardware prefetching in its various forms. Others have examined dynamic memory scheduling to help bridge the performance gap between processors and DRAM memory systems. This study builds on these results, combining a stride-based reference prediction table, a mechanism that prefetches L2 cache lines, and a memory controller that dynamically schedules accesses to a Direct Rambus memory subsystem. We find that such a system delivers good speedups for scientific applications with regular access patterns without negatively affecting the performance of nonstreaming programs. 1.

Cited by 0

NUMA-Aware Hardware Prefetching (SC '21, pap147): Artifacts

Sánchez Barrera, Isaac, Casas, Marc, Moretó, Miquel · Zenodo (CERN European Organization for Nuclear Research) · 2021

NUMA-Aware Hardware Prefetching (SC ’21, pap147): Artifacts Small updates in permissions and execution details. See README for further details.

Cited by 0Open access

Analysis of hardware prefetching across virtual page boundaries

Ronald Dreslinski, Али Саиди, Trevor Mudge et al. · 2007

Data cache prefetching in the L2 is at the forefront of pre-fetching research. In this paper we analyze the impact of virtual page boundaries on these prefetchers. Conservative measurements on real hardware show that 30-50% of consecutive virtual pages are mapped to pages which are not consecutive in physical memory. Advanced hardware prefetching techniques that detect access patterns which span virtual page boundaries often end up prefetching data that is from the wrong physical page. Meanwhile, current simulation techniques for evaluating prefetching algorithms assume that all virtual pages are mapped consecutively. We show that not accounting for virtual page boundaries in simulation can lead to overestimates of as much as 29% (9% on average). We also show that a simple prefetch filter can improve performance up to 32% (7% on average) and recover the overestimated performance. This leads to the conclusion that although previous simulations may not have accounted for virtual page boundaries, the results they demonstrate are still attainable and that it is not necessary to simulate virtual page boundaries to get accurate results. However, actual hardware designers should take care to implement a simple filter or else their hardware may not show the same gains in performance as they did in simulation.

Cited by 6

Multi-Strided Access Patterns to Boost Hardware Prefetching

M. Blom, Kristian F. D. Rietveld, Rob V. van Nieuwpoort · 2025

Important memory-bound kernels, such as linear algebra, convolutions, and stencils, rely on SIMD instructions as well as optimizations targeting improved vectorized data traversal and data re-use to attain satisfactory performance. On contemporary CPU architectures, the hardware prefetcher is of key importance for efficient utilization of the memory hierarchy. In this paper, we demonstrate that transforming a memory access pattern consisting of a single stride to one that concurrently accesses multiple strides, can boost the utilization of the hardware prefetcher, and in turn improves the performance of memory-bound kernels significantly. Using a set of micro-benchmarks, we establish that accessing memory in a multi-strided manner enables more cache lines to be concurrently brought into the cache, resulting in improved cache hit ratios and higher effective memory bandwidth without the introduction of costly software prefetch instructions. Subsequently, we show that multi-strided variants of a collection of six memory-bound dense compute kernels outperform state-of-the-art counterparts on three different micro-architectures. More specifically, for kernels among which Matrix Vector Multiplication, Convolution Stencil and kernels from PolyBench, we achieve significant speedups of up to 12.55x over Polly, 2.99x over MKL, 1.98x over OpenBLAS, 1.08x over Halide and 1.87x over OpenCV. The code transformation to take advantage of multi-strided memory access is a natural extension of the loop unroll and loop interchange techniques, allowing this method to be incorporated into compiler pipelines in the future.

Cited by 1Open access

NUMA-Aware Hardware Prefetching (SC '21, pap147): Artifacts

Sánchez Barrera, Isaac, Casas, Marc, Moretó, Miquel · Zenodo (CERN European Organization for Nuclear Research) · 2021

NUMA-Aware Hardware Prefetching (SC ’21, pap147): Artifacts Small updates in permissions and execution details. See README for further details.

Cited by 0Open access

A Hardware Prefetching Mechanism for Vector Gather Instructions

Hikaru Takayashiki, Masayuki Sato, Kazuhiko Komatsu et al. · 2019

Vector gather instructions are responsible for handling indirect memory accesses in vector processing. Since the indirect memory accesses usually express irregular access patterns, they have relatively low spatial and temporal locality compared with regular access patterns. As a result, an application with many vector gather instructions suffers from long latencies of the indirect memory accesses. Thus, the long latencies cause a significant performance degradation in vector processing. This paper proposes a hardware prefetching mechanism to hide memory access latencies of indirect memory accesses. The mechanism prefetches cacheable index data before executing a vector gather instruction, and predicts the addresses of the memory requests issued by the vector gather instruction. The mechanism then tries to prefetch the data based on the predicted addresses. As a result, the mechanism can reduce the memory access latencies of vector gather instructions. Moreover, this paper discusses how many cache blocks should be loaded per prediction regarding a single vector gather instruction by varying the prefetching parameters of distance and degree. In the evaluation, the performance of a simple kernel is examined with two types of index data: sequential and random. The evaluation results show that the prefetching mechanism improves the performance of the sequential-indexed and random-indexed kernels by 2.2x and 1.2x, respectively.

Cited by 3

Hardware-Only Stream Prefetching and Dynamic Access Ordering

Chengqiang Zhang, et al. · 2008

Memory system bottlenecks limit performance for many applications, and computations with strided access patterns are among the hardest hit. The streams used in such applications have extremely poor cache behavior. These access patterns have theadvantage of being predictable, though, and this can be exploited to improve the efficiency of the memory subsystem in two ways: memory latencies can be masked by prefetching stream data, and the latencies can be reduced by reordering stream accesses to exploit parallelism and locality within the DRAMs. Many researchers have studied hardware prefetching in its various forms. Others have examined dynamic memory scheduling to help bridge the performance gap between processors and DRAM memory systems. This study builds on these results, combining a stride-based reference prediction table, a mechanism that prefetches L2 cache lines, and a memory controller that dynamically schedules accesses to a Direct Rambus memory subsystem. We nd that such asystem delivers good speedups for scientific applications with regular access patterns without negatively affecting the performance of nonstreaming programs.

Cited by 0