Lecture 13 - Edward Rolls

---

Competitive Networks

Such networks (with a set of inputs and a winner take all output vector) work very well as feature detectors. How? Well, first you calculate the activation and then apply the winner-take-all algorithm. Notice that this has a way of condensing the input. Ie. reducing the original pattern into a sparse output signal. In essence, the cells are becoming "feature detectors" for a particular kind of input, much like a kind of cluster or principle components analysis.
     However, we wouldn't want one neuron to start capturing all of the patterns, so at the same time one synapse gets stronger another, inactive, synapse gets weaker (heterosynaptic LTD), thus the total synaptic strength remains constant. Indeed, neuroscientists had been finding, and worrying, over such effects for years, before connectionism came up with an idea for why this should be.

Properties

• Feature discovery by self-organisation
• Removal of redundancy (works well as a preprocessor to simplify the input)
• Orthogonalization and categorization (seems to be what is occuring with the CA3 dentate granule cells, which perform a kind of expansion function).
• Sparsification (In winner-take-all extreme, just one neuron wins)
• Capacity (depends on the number of outputs)
• Seperation of non-linearily seperable patterns

Topographic Maps

Can be produced by a special kind of competitive net called a Cohonen net, which is simply a net whose outputs have a very strong lateral inhibition function (mexican hat with short range excitation and long range inhibition). This has the effect of producing an output space in which related inputs cluster at output, even if mapping down several dimensions. Thus, such a net produces an optimal compression from one dimension to the next one down.
     Interestingly, such nets often have backward projections from the layer afterward. This has the effect of tuning earlier layers to correlations which turn out to be important much later in the net. Thus, multimodal coorelation later on can help tune earlier unimodal nets to the features important for that coorelation.

Reinforcement Learning

In the nets we have studied thus far, there haven't been any teachers, and yet, clearly there is some teaching occuring in the real world. (Eg. balancing a ruler on your hand) Sutton and Barto have proposed an Associative Reward-Penalty net, whose output produces a +1 if error + noise is greater than zero, -1 if anything else. Notice, however, that the noise occurs all the time and this effectively helps the network learn. Without mistakes, it never knows what makes things better.
     It turns out that such a net can also solve non-linearly seperable problems (like back propagation). Unfortunately, since the network only has one binary error signal, it is very slow to learn.