Thank you for this rather deep question. I will try to answer it the best I can.
Even though the Hopfield model has been very influential in computational neuroscience, it is a very simplistic model and we still don't know if it is a good model of what the brain does. This week lecture on the Hopfield model illustrates a general approach in computational neuroscience:
1). We identify cognitive task (in our case, auto-associative memory)
2). We find an algorithm that can solve this tasks (in our case, select the stored pattern with the minimal distance)
3). We give an neuronal implementation of this algorithm, with more or less biologically realistic neurons and synapses (in our case, a fully connected network with a basic Hebb rule for synaptic weights)
4). We confront this neuronal model with experimental data.
In the case of the Hopfield model, confronting the model with experimental data is challenging and is ongoing research. This is why I say that we don't know if it is a good model of the brain yet.
Regarding your question on Hebbian learning, the synaptic weights in the Hopfield model follow a very simple Hebb rule: if you take the example of the "swiss cross" network of the lecture, black neurons are connected to other black neurons with weight +1, white neurons are connected to other white neurons with weight +1 but the synaptic weight between neurons of different color is -1. This can be seen as a rudimentary example of "fire together, wire together". (Black neurons, which respond to the color black wire together, the same for white neurons.)
If things are still not clear, feel free to ask me a question during the next exercise session.
Thank you for your answer.
So being a data science student if I was asked to find an algorithm that would take in an image or other data and tell me what it is (or what it is closely resembling) I would have used a neural network or a convolutional neural network (apparently combining steps 2 and 3 ?). Which also happens to be inspired by the brain/visual cortex. would that be another acceptable way to model the associate memory ?
The Hopfield model is not doing pattern recognition or classification. It only does pattern completion: you give a noisy image or an image with missing pixels as input and it outputs the original (stored) image. Hence, the Hopfield model and artificial neural networks (ANN) do not solve the same problem.
ANN are not meant to model associative memory.
A question closely related to your question is "what does the brain do when it recognizes an object in an image ?". Pattern classification ? Pattern completion ? Some scientists have focused on the ANN perspective (James DiCarlo for example) and others on the Hopfield network perspective (Daniel Amit and his former students/collaborators for example). These questions are still open and subject of intensive investigation.
Follow the pulses of EPFL on social networks
© 2023 EPFL, all rights reserved