Topology optimization in spatially distributed cellular neural network
Date
2012
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
A new network topology optimization approach to cellular neural network design, as a method for realizing associative memories using sparser networks is conceptualized. This type of optimization allows recurrent neural networks to be implemented in a spatially distributed fashion, that is, with components of the network residing in different physical locations. This could find application in addressing the problem of dynamic allocation of a team of robots to a collection of spatially distributed tasks which is relevant for large scale environmental monitoring and surveillance. Spatially distributed sensing allows for greater coverage of the environment than a single large vehicle with multiple sensors would permit in many cases. In this work, we try to answer the question of how could the design process be different if the network topology was also part of the design. A sparser cellular neural network topology can be achieved without significantly degrading the performance of the network, by selectively deleting those weights from the optimized network which contribute the least to ability of the network to recall the desired patterns. This approach is particularly useful where neural links incur varying costs, such as implementation of associative memories over wireless sensor networks. The cellular neural networks interconnection topology is diluted, without significantly degrading its performance, where performance is quantified by the average recall probability of the patterns engraved into the networks associative memory. The average recall probability is a measure of performance of the designed network in presence of noise and is defined as the ratio of number of recovered memory patterns (perturbed initial condition vectors which result in same output as the stored memory vector) to the total number of perturbed initial condition vectors. Since the average recall probability cannot be assessed prior to testing, the optimization algorithm uses the networks stability parameters as a measure of quality of memorization, and optimization proceeds by selectively removing costly links that contribute the least to the magnitude of these parameters. Two different approaches to implementing the optimization of the networks topology are implemented and compared. The first one is a sequential process in which a single link is removed each time, specifically the one the removal of which incurs the least performance cost compared to all other existing high-cost links. This method ignores the possibility that a non-obvious combination of links may produce better results through the links simultaneous removal. This phenomenon has been observed in simulation studies which validated the proposed method. To validate further the optimization, but more importantly, to ensure that the overall approach does not depend on the particular method used for the combinatorial optimization we also implemented an alternative approach which is based on the randomized optimization. In this approach a random sample of a sufficient number of i.i.d possible topology is generated. In other words, each random topology in the sample has the same probability distribution as the others and all are mutually independent. An example is used to demonstrate that irrespectively of the combinatorial algorithm used, the approach yields sparser associative memories that in general trade off performance for cost, and in many cases the performance of the diluted network is on par with the original system. In our numerical tests, the two methods yield comparable results, which do not differ significantly in terms of resulting network performance. Performance is quantified in terms of the network recall probability, and in the proposed optimization algorithm approach is captured by the neural networks stability parameters. Further, we apply the ideas developed so far to control network communication in actual robots to experimentally verify our simulation results. Experimental testing has shown that spatially distributed implementations of cnn on CoroBots are indeed feasible, and that for some cases, the communication delays related to the communication between the different components of the network are not significant enough to affect the performance and stability properties of the dynamical system. It is shown that the error between simulation of the discrete-time dynamics and experimental results practically coincide, with a maximum error difference of the order of 10-4. Thus the proposed combinatorial optimization methods performed almost equally well in practice as in simulations.