Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

preprint · arXiv (Cornell University) · 2023

Training Convolutional Neural Networks with the Forward-Forward algorithm

Riccardo Scodellaro, Ajinkya Kulkarni, Frauke Alves, Matthias Schröter

Cited by 3Open access

Abstract

Recent successes in image analysis with deep neural networks are achieved almost exclusively with Convolutional Neural Networks (CNNs), typically trained using the backpropagation (BP) algorithm. In a 2022 preprint, Geoffrey Hinton proposed the Forward-Forward (FF) algorithm as a biologically inspired alternative, where positive and negative examples are jointly presented to the network and training is guided by a locally defined goodness function. Here, we extend the FF paradigm to CNNs. We introduce two spatially extended labeling strategies, based on Fourier patterns and morphological transformations, that enable convolutional layers to access label information across all spatial positions. On CIFAR10, we show that deeper FF-trained CNNs can be optimized successfully and that morphology-based labels prevent shortcut solutions on dataset with more complex and fine features. On CIFAR100, carefully designed label sets scale effectively to 100 classes. Class Activation Maps reveal that FF-trained CNNs learn meaningful and complementary features across layers. Together, these results demonstrate that FF training is feasible beyond fully connected networks, provide new insights into its learning dynamics and stability, and highlight its potential for neuromorphic computing and biologically inspired learning.

Cite this paper

APA 7

Scodellaro, R., Kulkarni, A., Alves, F., & Schröter, M. (2023). Training Convolutional Neural Networks with the Forward-Forward algorithm. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2312.14924

MLA 9

Scodellaro, Riccardo, et al. “Training Convolutional Neural Networks with the Forward-Forward algorithm.” arXiv (Cornell University), 2023, https://doi.org/10.48550/arxiv.2312.14924.

Chicago

Scodellaro, Riccardo, Ajinkya Kulkarni, Frauke Alves, and Matthias Schröter. 2023. “Training Convolutional Neural Networks with the Forward-Forward algorithm.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2312.14924.

Harvard

Scodellaro, R. et al. (2023) ‘Training Convolutional Neural Networks with the Forward-Forward algorithm’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.2312.14924.

BibTeX
@misc{scodellaro2023training,
  title = {{Training Convolutional Neural Networks with the Forward-Forward algorithm}},
  author = {Scodellaro, Riccardo and Kulkarni, Ajinkya and Alves, Frauke and Schröter, Matthias},
  journal = {arXiv (Cornell University)},
  year = {2023},
  doi = {10.48550/arxiv.2312.14924}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.

Related papers