How to utilize Hebbian learning?
I want to upgrade my evolution simulator to use Hebb learning, like this one. I basically want small creatures to be able to learn how to find food. I achieved that with the basic feedforward networks, but I'm stuck at understanding how to do it with Hebb learning. The basic principle of Hebb learning is that, if two neurons fire together, they wire together.
So, the weights are updated like this:
weight_change = learning_rate * input * output
The information I've found on how this can be useful is pretty scarce, and I don't get it.
In my current version of the simulator, the weights between an action and an input (movement, eyes) are increased when a creature eats a piece of food, and I fail to see how that can translate into this new model. There simply is no room to tell if it did something right or wrong here, because the only parameters are input and output! Basically, if one input activates movement in one direction, the weight would just keep on increasing, no matter if the creature is eating something or not!
Am I applying Hebb learning in a wrong way? Just for reference, I'm using Python.
Answer
Hebbs law is a brilliant insight for associative learning, but its only part of the picture. And you are right, implemented as you have done, and left unchecked a weight will just keep on increasing. The key is to add in some form of normalisation or limiting process. This is illustrated quite well of the wiki page for Oja's rule. What I suggest you do is add in a post-synaptic divisive normalisation step, what this means is that you divide through a weight by the sum of all the weights converging on the same post-synaptic neuron (i.e. the sum of all weights converging on a neuron is fixed at 1).
What you want to do can be done by building a network that utilises Hebbian learning. I'm not quite sure on what you are passing in as input into your system, or how you've set things up. But you could look at LISSOM which is an Hebbian extension to SOM, (self-organising map).
In a layer of this kind typically all the neurons may be interconnected. You pass in the input vector, and allow the activity in the network to settle, this is some number of settling steps. Then you update the weights. You do this during the training phase, at the end of which associated items in the input space will tend to form grouped activity patches in the output map.
It's also worth noting that the brain is massively interconnected, and highly recursive (i.e. there is feedforward, feedback, lateral interconnectivity, microcircuits, and a lot of other stuff too..).