Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

article · Scholarly Commons (University of Pennsylvania) · 2000

An Information Maximization Approach to Overcomplete and Recurrent Representations

Oren Shriki, Haim Sompolinsky, Daniel D. Lee

Cited by 22Open access

Abstract

The principle of maximizing mutual information is applied to learning overcomplete and recurrent representations. The underlying model consists of a network of input units driving a larger number of output units with recurrent interactions. In the limit of zero noise, the network is deterministic and the mutual information can be related to the entropy of the output units. Maximizing this entropy with respect to both the feedforward connections as well as the recurrent interactions results in simple learning rules for both sets of parameters. The conventional independent components (ICA) learning algorithm can be recovered as a special case where there is an equal number of output units and no recurrent connections. The application of these new learning rules is illustrated on a simple two-dimensional input example.

Cite this paper

APA 7

Shriki, O., Sompolinsky, H., & Lee, D. D. (2000). An Information Maximization Approach to Overcomplete and Recurrent Representations. Scholarly Commons (University of Pennsylvania), 13, 612–618.

MLA 9

Shriki, Oren, et al. “An Information Maximization Approach to Overcomplete and Recurrent Representations.” Scholarly Commons (University of Pennsylvania), vol. 13, 2000, pp. 612–18.

Chicago

Shriki, Oren, Haim Sompolinsky, and Daniel D. Lee. 2000. “An Information Maximization Approach to Overcomplete and Recurrent Representations.” Scholarly Commons (University of Pennsylvania) 13: 612–18.

Harvard

Shriki, O., Sompolinsky, H. and Lee, D.D. (2000) ‘An Information Maximization Approach to Overcomplete and Recurrent Representations’, Scholarly Commons (University of Pennsylvania), 13, pp. 612–618.

BibTeX
@article{shriki2000information,
  title = {{An Information Maximization Approach to Overcomplete and Recurrent Representations}},
  author = {Shriki, Oren and Sompolinsky, Haim and Lee, Daniel D.},
  journal = {Scholarly Commons (University of Pennsylvania)},
  volume = {13},
  pages = {612--618},
  year = {2000}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.

Related papers