Sparse Coding (Olshausen & Field, 1996) Model

Sparse Coding (Olshausen & Field, 1996) Model with LCA (locally competitive algorithm) (Rozell et al., Neural Comput. 2008).

Data is from http://www.rctn.org/bruno/sparsenet/.

Requirement

• Python >= 3.5
• numpy, matplotlib, scipy, tqdm, sklearn

Usage

• Run train.py or predictive-coding.ipynb (written in Japanese).
• ica.py is implementation of ICA and PCA for Natural images.

Results

Loss function

Receptive fields (using soft threshold function)

Receptive fields (using Cauchy threshold function (Mayo et al., 2020))

ICA

PCA

Reference

• Olshausen BA, Field DJ. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature. 1996;381(6583):607–609. Data and Code, pdf
• Rozell CJ, Johnson DH, Baraniuk RG, Olshausen BA. Sparse coding via thresholding and local competition in neural circuits. Neural Comput. 2008;20(10):2526‐2563.
• Mayo P, Holmes R, Achim A. Iterative Cauchy Thresholding: Regularisation with a heavy-tailed prior. arXiv. 2020.