Antony W. N’dri, Thomas Barbier, Céline Teulière et al. · Nature Communications · 2025
Current machine learning systems consume vastly more energy than biological brains. Neuromorphic systems aim to overcome this difference by mimicking the brain's information coding via discrete voltage spikes. However, it remains unclear how both artificial and natural networks of spiking neurons can learn energy-efficient information processing strategies. Here we propose Predictive Coding Light (PCL), a recurrent hierarchical spiking neural network for unsupervised representation learning. In contrast to previous predictive coding approaches, PCL does not transmit prediction errors to higher processing stages. Instead it suppresses the most predictable spikes and transmits a compressed representation of the input. Using only biologically plausible spike-timing based learning rules, PCL reproduces a wealth of findings on information processing in visual cortex and permits strong performance in downstream classification tasks. Overall, PCL offers a new approach to predictive coding and its implementation in natural and artificial spiking neural networks.
Cited by 1Open access
SpringerReference · 2011
Cited by 11
Martin H. Weik · 2000
Cited by 0
Sven Treitel · Science · 1979
Cited by 0
CHAVY, Cyril, ZARADER, Jean-Luc · OpenGrey (Institut de l'Information Scientifique et Technique) · 2004
Cited by 0Open access
Peter A. van der Helm · Psychonomic Bulletin & Review · 2015
Cited by 15Open access
Yanping Huang, Rajesh P. N. Rao · 2015
Predictive coding is aunifying framework for understanding redundancy reduction and efficient coding in the nervous system. By transmitting only the unpredicted portions of an incoming sensory signal, predictive coding allows the nervous system to reduce redundancy and make full use of the limited dynamic range of neurons. Starting with the hypothesis of efficient coding as a design principle in the sensory system, predictive coding provides a functional explanation for a range of neural responses andmany aspects of brain organization. The lateral and temporal antagonism in receptive fields in the retina and lateral geniculate nucleus occur naturally as a consequence of predictive coding of natural images. In the higher visual system, predictive coding provides an explanation for oriented receptive fields and contextual effects as well as the hierarchical reciprocally connected orga-nization of the cortex. Predictive coding has also been found to be consistent with a variety of neurophysiological and psychophysical data obtained from different
Cited by 0
Paweł Gładziejewski · Synthese · 2015
According to the predictive coding theory of cognition (PCT), brains are predictive machines that use perception and action to minimize prediction error, i.e. the discrepancy between bottom–up, externally-generated sensory signals and top–down, internally-generated sensory predictions. Many consider PCT to have an explanatory scope that is unparalleled in contemporary cognitive science and see in it a framework that could potentially provide us with a unified account of cognition. It is also commonly assumed that PCT is a representational theory of sorts, in the sense that it postulates that our cognitive contact with the world is mediated by internal representations. However, the exact sense in which PCT is representational remains unclear; neither is it clear that it deserves such status—that is, whether it really invokes structures that are truly and nontrivially representational in nature. In the present article, I argue that the representational pretensions of PCT are completely justified. This is because the theory postulates cognitive structures—namely action-guiding, detachable, structural models that afford representational error detection—that play genuinely representational functions within the cognitive system.
Cited by 184Open access
Jagiellonian University Repository (Jagiellonian University) · 2018
Cited by 0
Sarah Conroy Stephen O'Connor · 2015
Cited by 0
Howard Bowman, Dan Collins, A. Nayak et al. · Neuroscience & Biobehavioral Reviews · 2023
Predictive-coding has justifiably become a highly influential theory in Neuroscience. However, the possibility of its unfalsifiability has been raised. We argue that if predictive-coding were unfalsifiable, it would be a problem, but there are patterns of behavioural and neuroimaging data that would stand against predictive-coding. Contra (vanilla) predictive patterns are those in which the more expected stimulus generates the largest evoked-response. However, basic formulations of predictive-coding mandate that an expected stimulus should generate little, if any, prediction error and thus little, if any, evoked-response. It has, though, been argued that contra (vanilla) predictive patterns can be obtained if precision is higher for expected stimuli. Certainly, using precision, one can increase the amplitude of an evoked-response, turning a predictive into a contra (vanilla) predictive pattern. We demonstrate that, while this is true, it does not present an absolute barrier to falsification. This is because increasing precision also reduces latency and increases the frequency of the response. These properties can be used to determine whether precision-weighting in predictive-coding justifiably explains a contra (vanilla) predictive pattern, ensuring that predictive-coding is falsifiable.
Cited by 24Open access
Daniel R. Williams · Synthese · 2018
Predictive processing has recently been advanced as a global cognitive architecture for the brain. I argue that its commitments concerning the nature and format of cognitive representation are inadequate to account for two basic characteristics of conceptual thought: first, its generality—the fact that we can think and flexibly reason about phenomena at any level of spatial and temporal scale and abstraction; second, its rich compositionality—the specific way in which concepts productively combine to yield our thoughts. I consider two strategies for avoiding these objections and I argue that both confront formidable challenges.
Cited by 61Open access
N'dri, Antony W., Triesch, Jochen, Teulière, Céline et al. · Zenodo (CERN European Organization for Nuclear Research) · 2025
Programming code / software of the paper "Predictive Coding Light" accepted in Nature Communications.
Cited by 0Open access
Rui Meng, T. David Luo, Kristofer E. Bouchard · arXiv (Cornell University) · 2022
Unsupervised learning plays an important role in many fields, such as artificial intelligence, machine learning, and neuroscience. Compared to static data, methods for extracting low-dimensional structure for dynamic data are lagging. We developed a novel information-theoretic framework, Compressed Predictive Information Coding (CPIC), to extract useful representations from dynamic data. CPIC selectively projects the past (input) into a linear subspace that is predictive about the compressed data projected from the future (output). The key insight of our framework is to learn representations by minimizing the compression complexity and maximizing the predictive information in latent space. We derive variational bounds of the CPIC loss which induces the latent space to capture information that is maximally predictive. Our variational bounds are tractable by leveraging bounds of mutual information. We find that introducing stochasticity in the encoder robustly contributes to better representation. Furthermore, variational approaches perform better in mutual information estimation compared with estimates under a Gaussian assumption. We demonstrate that CPIC is able to recover the latent space of noisy dynamical systems with low signal-to-noise ratios, and extracts features predictive of exogenous variables in neuroscience data.
Cited by 0Open access
Markus J. Hofmann A, Michael Dambacher D, Arthur M. Jacobs B et al. · 2016
Cited by 0
SpringerReference · 2011
Cited by 0
Rakesh Chalasani, José C. Prı́ncipe · arXiv (Cornell University) · 2013
The quality of data representation in deep learning methods is directly related to the prior model imposed on the representations; however, generally used fixed priors are not capable of adjusting to the context in the data. To address this issue, we propose deep predictive coding networks, a hierarchical generative model that empirically alters priors on the latent representations in a dynamic and context-sensitive manner. This model captures the temporal dependencies in time-varying signals and uses top-down information to modulate the representation in lower layers. The centerpiece of our model is a novel procedure to infer sparse states of a dynamic model which is used for feature extraction. We also extend this feature extraction block to introduce a pooling function that captures locally invariant representations. When applied on a natural video data, we show that our method is able to learn high-level visual features. We also demonstrate the role of the top-down connections by showing the robustness of the proposed model to structured noise.
Cited by 33Open access
Mark Sprevak · Philosophy Compass · 2023
Abstract Predictive coding – sometimes also known as ‘predictive processing’, ‘free energy minimisation’, or ‘prediction error minimisation’ – claims to offer a complete, unified theory of cognition that stretches all the way from cellular biology to phenomenology. However, the exact content of the view, and how it might achieve its ambitions, is not clear. This series of articles examines predictive coding and attempts to identify its key commitments and justification. The present article begins by focusing on possible confounds with predictive coding: claims that are often identified with predictive coding, but which are not predictive coding. These include the idea that the brain employs an efficient scheme for encoding its incoming sensory signals; that perceptual experience is shaped by prior beliefs; that cognition involves minimisation of prediction error; that the brain is a probabilistic inference engine; and that the brain learns and employs a generative model of the world. These ideas have garnered widespread support in modern cognitive neuroscience, but it is important not to conflate them with predictive coding.
Cited by 11Open access
N'dri, Antony W., Triesch, Jochen, Teulière, Céline et al. · Zenodo (CERN European Organization for Nuclear Research) · 2025
Programming code / software of the paper "Predictive Coding Light" accepted in Nature Communications.
Cited by 0Open access
Veronika Koren (3691690), Sophie Denève (355889) · Figshare · 2017
<p>A: Auto-encoder (schema). Single neuron with weight <i>w</i> = 1 represents the prediction error within its membrane potential. Whenever the distance between the signal (blue) and its estimate (red) is bigger than the half of neuron’s weight, neuron spikes, which pulls the estimate towards the signal. B: Four neurons, two with weight +10 and two with weight -10, track a slow oscillating signal (gray on the upper plot). When the estimate (black on the upper plot) is too far from the signal from below, one of + neurons fires a spike (pink dots) to pull the estimate up. When the estimate is too far from the signal from above, one of—neurons fires (blue dots) to pull it down. The membrane potentials of + and - neurons are in anti-phase (lower plot). The order of firing within the + and - group is irrelevant for the read-out of spiking activity. Parameters: <i>ν</i> = 1, <i>μ</i> = 0, <i>σ</i> = 1, <i>λ</i> = 4 C: Tracking the same signal as in b) with 50 neurons, 25 with weight +0.1 and 25 with weight -0.1, gives rise to asynchronous spike trains (raster plot). Since neurons now have smaller weights and, as a network, fire more spikes, the estimation of the signal (upper plot) is much more precise. Parameters: <i>ν</i> = 1, <i>μ</i> = 0, <i>σ</i><sub><i>thres</i></sub> = 0.1, <i>λ</i> = 4 Hz. D: The “ping-pong” effect. 4 neurons, 2 with weight +1 and 2 with weight -1, do not receive any feed-forward drive but receive uncorrelated white noise in their membrane potentials. When one neuron fires a spike because of the noise, this activates lateral connections, provoking other spikes (middle plot). A spike of a + neuron is followed by a spike of a—neuron, which creates fast oscillation in the estimate (upper plot) as well as in the membrane potentials (lower plot). Parameters: <i>ν</i> = 1, <i>μ</i> = 0, <i>σ</i> = 0.1, <i>λ</i> = 4 Hz.</p>
Cited by 0Open access