[Solved]: How to determine if a state is a fixed point in a Hopfield network?

Problem Detail: 

I have been reading a lot and I am still unsure of how to determine this. Let's say I have an initial binary state vector (1, 1, 1). How would I go about determining whether (1, 1, 1) is a fixed point in the network?

My initial understanding was to put the state through the network and if it converged to (1, 1, 1) then it is a fixed point. However I have a feeling it is not as simple as that.

Thanks.

Asked By : Kevin Lee

Answered By : alto

Your intuition is right. A state is a fixed point in a Hopfield network if it is a local minima of the energy function. In other words, some binary state vector $x$ is a fixed point if and only if every other state vector at hamming distance $1$ from $x$ has higher energy. The simplest way to determine if a state is a fixed point is to iterate over each dimension of the state and try to update it. If the state doesn't change then it is a fixed point.

Edit: Too long for comment.

I think you might be confusing global and local minima. If you take some state (binary vector) $\mathbf{x} = (x_1, \ldots, x_n)$, and iterate the dynamics of the network until nothing changes$^{1}$, you will have converged on a fixed point, call it $\mathbf{x}^\prime$. This $\mathbf{x}^\prime$ is a local minima of the energy function. By definition this means that every state at hamming distance 1 (those that differ by 1 bit) will have higher energy. If this was not the case then there exists some $i \in \{1,\ldots,n\}$ such that flipping position $i$ in $\mathbf{x}^\prime$ would give a state with lower energy. But this means when choosing to update bit $i$, you would have flipped it$^2$.

Now, what this doesn't mean is the that this is the global minima of the energy function. There could be a state that differs by 2 bits from $\mathbf{x}^\prime$ with lower energy. But you can never reach it from $\mathbf{x}^\prime$ by updating a single bit/unit at a time. So you are trapped at $\mathbf{x}^\prime$, in a local minima of the energy function.

[1] This is important, you need to make a full pass over all the postions in the state vector without anything changing to ensure you have converged to a fixed point.

[2] The relation between the energy function and the dynamics of asynchronously updating units is what makes Hopfield networks interesting. The idea that we can say something global about the network (the energy function) based on local interactions is neat.

Best Answer from StackOverflow

Question Source : http://cs.stackexchange.com/questions/23969

3.2K people like this

 Download Related Notes/Documents