Chaotic Neural Network Models
``Brain, the most complicated kilogram
of matter in the universe''
- Anonymous
I am interested in the role of nonlinear dynamics (especially chaos) in the functioning of the brain. To this end, my approach had been a kind of ``reverse-engineering'' - building simple neural network models exhibiting chaotic dynamics and using them for various information processing tasks. The simplicity allows a theoretical analysis to be done; in addition, if such simplified models can show advantages in performing various sensory processing tasks, one can argue that the more complicated hardware of the brain should also be using their inherent nonlinear dynamics for similar ends.
The simplest model that can show an interesting range of dynamical behavior is a pair of coupled excitatory and inhibitory neurons, evolving in discrete time-intervals (see figure below).
 |
The 1-dimensional neural map, F. The activation functions for the constituent excitatory (slope, a = 20) and inhibitory (b = 5) neurons are also plotted using broken lines. The excitatory (x)-inhibitory (y) neural pair is shown as an inset. Links terminating in arrows indicate excitatory connections, while those terminating in circles indicate inhibitory connections. Wxx and Wyy are self-connectances ( to account for delays) and Wxy and Wyx are inter-connection weights between the excitatory and inhibitory neurons. |
The time-evolution of the system is given by the pair of equations:
x n+1 = F a ( W xx x n
- W xy y n ),
y n+1 = F b ( W yx x n - W
yy y n ),
where x n, y n are the mean firing rates of excitatory and inhibitory neurons, respectively, at the n-th time step. F a,b are the neural activation functions which specify the nature of the input-output conversion. It may be of piecewise linear nature:
| F a ( z ) = | 0, if z < 0, |
|
= |
a z , if 0 < z < 1/a, |
|
= |
1 if z > 1/a. |
Otherwise, they might also be of a sigmoidal nature:
F a ( z ) = 1 / (1 + exp (- a z)).
Both types of activation functions show chaotic as well as periodic and fixed-point behavior.