Ph.D. thesis of
Indian Statistical Institute ,
Calcutta
submitted: Nov 1998, awarded: Jan 2000
Abstract
Artificial neural network models have grown up on the premise that the massively parallel distributed processing and connection ist structure observed in the brain is the key behind its superior performance. By incorporating these features in the design of a new class of computer architectures and algorithms, it is hoped that, machines will exhibit human-like ability in handling real-world situations. Neural network models comprise a large number of threshold activated elements connected to each other. A network is parametrized by its connection topology, characteristics of the threshold elements, and the learning rules for computing the connection weights between the elements. The conventional neural network paradigm has centered around the ``fixed-point'' approach, where, the dynamics involves gradient descent of the network state to stable fixed-points (or, attractors of period 0) corresponding to desired patterns. However, human memory is an extremely dynamic phenomenon. Recent neurobiological evidence indicates the dominance of chaotic activity in brain processes. At present, performance of the conventional neural models is very much limited when compared to that of the human brain. To overcome such limitations, the model should incorporate significant biological information so far not taken into account. We have therefore undertaken an extensive study of chaotic dynamics in neural models, to evaluate its possible functional significance in brain performance.
The present thesis reports some results of investigatio n on the behavior of simple excitatory-inhibitory network models. Almost throughout, a strict form of Dale's hypothesis (according to which, a neuron has exclusively excitatory or inhibitory synaptic connections) is assumed. The resultant discrete-time dynamics (with synchronous updating) has shown a variety of interesting features. The underlying motivation is to look at the simplest neural module capable of showing chaotic behavior and to use the knowledge gained from studying this system to obtain a broader understanding of the possible relevance of chaotic dynamics to brain functioning . A pair of excitatory and inhibitory neurons, coupled to each other, appears to be the simplest system which shows the desired behavior. Detailed theoretical and simulation study has therefore been conducted on this module, which has later on been used as the building block for constructing more complex networks. Several types of nonlinear activation functions (having unimodal/bimod al and sigmoid/piecewise linear characteristics) have been used in order to find out features that are generic and those that are specific to a certain type of function. The response of the module to external stimulus, both constant and periodic in time, has been studied. The observed dynamical transitions have been analyzed and for a range of parameter values, the response of the model exhibits features remarkably similar to ``stochastic resonance''. The control of chaotic behavior of the neural model has been shown to be possible with both external periodic forcing as well as occasional proportional feedback to the system variable. A brief physical explanation of the control mechanism is provided. The collective dynamics, and in particular , synchronization among small assemblies of coupled modules, have been investigate d with particular emphasis on the role of competing interactions. Finally, the possibility of exploiting such networks for visual information processing, especially for image segmentation and adaptive smoothing, has been examined.
Click on the following links to download the thesis chapter by chapter as pdf files:
Front matter and Chapter 1 (Introduction)
Chapter 2 (Intrinsic Dynamics of an Excitatory-Inhibitory Neural Pair)
Chapter 3 (Nonlinear Resonance in a Chaotic Neural Pair)
Chapter 4 (Chaos Control in Simple Excitatory-Inhibitory Neural Network Models)
Chapter 5 (Collective Dynamics and Synchronization in Small Assemblies of Neural Pairs)
Chapter 6 (Visual Information Processing with Excitatory-Inhibitory Networks)
Chapter 7 (Conclusions) and Bibliography