Associative Memory in a Continuous Metric Space: A Theoretical Foundation

Abstract

Abstract We introduce a formal theoretical background, which includes theorems and their proofs, for a neural network model with associative memory and continuous topology, i.e. its processing units are elements of a continuous metric space and the state space is Euclidean and infinite dimensional. This approach is intended as a generalization of the previous ones due to Little and Hopfield. The main contribution of the present work is to integrate -and to provide a theoretical background that makes this integration consistent- two levels of continuity: i) continuous response processing units and ii) continuous topology of the neural system, obtaining a more biologically plausible model of associative memory. We present our analysis according to the following sequence of steps: general results concerning attractors and stationary solutions, including a variational approach for the derivation of the energy function; focus on the case of orthogonal memories, proving theorems on stability, size of attraction basins and spurious states; considerations on the problem of resolution, analyzing the more general case of memories that are not orthogonal, and with possible modifications to the synaptic operator; getting back to discrete models, i. e. considering new viewpoints arising from the present continuous approach and examine which of the new results are also valid for the discrete models; discussion about the generalization of the non deterministic, finite temperature dynamics.