Wiener Filter

To develop an algorithm we could try to aim to find an optimum weight vector $\ve{w}_opt$ such that minimizes the mean squared error of the output signal $y$ with respect to the reference $d$.

For a single neuron without activation function, we can formulate the output of neuron $k$ as:

$$y(n) = v(n) = \ve{w} \ve{x}_n$$

So we can formulate the mean square error as:

$$\mathcal{E}(\ve{w}) = E\left\lbrace \vert d(n) - y(n)\vert^2... ...left\lbrace \vert d(n) - \ve{w} \ve{x}_n \vert^2 \right\rbrace$$

Minimizing the error⁶:

$$\nabla \mathcal{E}(\ve{w}_{opt}) = \ve{0} \quad \Rightarrow ... ... \esp{\ve{x}\ve{x}^h} \ve{w}_{opt} - \esp{d(n)\ve{x}} = \ve{0}$$

$$\ve{w}_{opt} = \esp{\ve{x}\ve{x}^h}^{-1} \esp{d(n)\ve{x}}$$

Where $\esp{\ve{x}\ve{x}^h}^{-1}$ is the autocorrelation matrix of the input. If we state $\ma{R}_x = \esp{\ve{x}\ve{x}^h}$ and $\ve{P} = \esp{d(n)\ve{x}}$, which we will call P-vector. We can write:

$$\ve{w}_{opt} = \ma{R}_x^{-1} \ve{P}$$