Measuring error

The mean square error between the output and the reference it's called in Neural networks literature Risk functional, but it's the same as the mean power of the error signal $\ve{e}$, the error is defined as:

$$\ve{e} = \ve{d} -\ve{y}$$

Where $\ve{y}$ is the output of the neural network and $\ve{d}$ is the reference signal or desired response. A scheme of supervised learning wich is used in the peceptron, the Wiener-Hopf filter, the gradient-descent and derived methods is:

All these methods are based in the same principle, minimize the risk functional or the mean squared error, which is:

$$R_w = E\left\lbrace \vert\vert\ve{e}\vert\vert^2 \right\rbrace = \mathcal{E}_w$$

This will often be impossible to get, so it would be estimated by $\hat{R}(w)$, (the training error) as:

$$\hat{R}_w = \frac{1}{N} \sum_{i=0}^{N-1} (\ve{d}_i - \ve{y}_i)$$

Where $N$ is the number of available learning pairs of desired response and actual response of the network.