article · Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics · 2003
Learning curves for mutual information maximization
Cited by 2Open access
Abstract
An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed [S. Becker and G. Hinton, Nature (London) 355, 161 (1992)]. For a generic data model, I show that in the large sample limit the structure in the data is recognized by mutual information maximization. For a more restricted model, where the networks are similar to perceptrons, I calculate the learning curves for zero-temperature Gibbs learning. These show that convergence can be rather slow, and a way of regularizing the procedure is considered.
Cite this paper
Urbanczik, R. (2003). Learning curves for mutual information maximization. Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics, 68(1), 016106. https://doi.org/10.1103/physreve.68.016106
Urbanczik, Robert. “Learning curves for mutual information maximization.” Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics, vol. 68, no. 1, 2003, p. 016106, https://doi.org/10.1103/physreve.68.016106.
Urbanczik, Robert. 2003. “Learning curves for mutual information maximization.” Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics 68 (1): 016106. https://doi.org/10.1103/physreve.68.016106.
Urbanczik, R. (2003) ‘Learning curves for mutual information maximization’, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics, 68(1), p. 016106. Available at: https://doi.org/10.1103/physreve.68.016106.
@article{urbanczik2003learning,
title = {{Learning curves for mutual information maximization}},
author = {Urbanczik, Robert},
journal = {Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics},
volume = {68},
number = {1},
pages = {016106},
year = {2003},
doi = {10.1103/physreve.68.016106}
}Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.
Related papers
[Implementation of mutual information based medical image registration methods].
Zhiyong Gao, Bin Gu, Jia‐Rui Lin · PubMed · 2003
Cited by 3
Monomodal image registration using mutual information based methods
Zhiyong Gao, Bin Gu, Jiarui Lin · Image and Vision Computing · 2006
Cited by 56
A New Criterion of Mutual Information Using R-value
Taegoon Han, Youngbok Hong, Sejong Oh · Lecture notes in electrical engineering · 2013
Cited by 0
Mutual information maximization for improving and interpreting multi-layered neural networks
Ryotaro Kamimura · IEEE Symposium Series on Computational Intelligence (SSCI) · 2017
Cited by 9
Self-Organized Mutual Information Maximization Learning for Improved Generalization Performance
Ryotaro Kamimura · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · 2015
Cited by 4
Supposed Maximum Mutual Information for Improving Generalization and Interpretation of Multi-Layered Neural Networks
Ryotaro Kamimura · Journal of Artificial Intelligence and Soft Computing Research · 2018
Cited by 14Open access
Collective mutual information maximization to unify passive and positive approaches for improving interpretation and generalization
Ryotaro Kamimura · Neural Networks · 2017
Cited by 6
An Adaptive Neighborhood Partition Full Conditional Mutual Information Maximization Method for Feature Selection
Gaoshuai Wang, Fabrice Lauri, Pu Wang et al. · arXiv (Cornell University) · 2022
Cited by 2Open access