Associative memory

$\ma{W}_m$ is the weight matrix of the $m^{th}$ pattern association that maps $\ve{x}_m \to \ve{y}_m$ defined as:

$$\ma{W}_m = \begin{pmatrix} w_{00}[m] & w_{01}[m] & \hdots &... ...\ w_{m0}[m] & w_{m1}[m] & \hdots & w_{mm}[m]\\ \end{pmatrix}$$

Where the neurons have no bias, and the index $m$ denotes the $m^{th}$ pattern association. The output vector $\ve{y}_m$ is given by the following equation in matrix notation:

$$\ve{y}_m = \ma{W}_m \ve{x}_m$$

The memory matrix $\ma{M}$ of $N$ pattern associations is defined as:

$$\ma{M} = \sum_{n=0}^{N-1} \ma{W}_n$$

Or with the recurrent relation:

$$\ma{M}_n = \ma{M}_{n-1} + \ma{W}_n$$

Which can be estimated as the sum of outer products of $\vec{y}$ and $\vec{x}$:

$$\ma{\hat{M}} = \sum_{n=0}^{N-1} \ve{y}_n \ve{x}_n^t = (\ve{y}... ...x}_1\\ \vdots\\ \ve{x}_{N-1} \end{pmatrix}= \ma{Y}\ma{X}^t$$

Or in a recursive way:

$$\ma{\hat{M}}_n = \ma{\hat{M}}_{n-1} + \ve{y}_n \ve{x}_n^t$$

So $\ve{y}_n \ve{x}_n^t = \ma{\hat{W}}_n$ is an estimator of the weight matrix.
The memory associates perfectly when the set of input vectors is orthonormal, the upper bound of patterns that can be associated is the dimension of $\ma{M}$.