Universal approximation theorem

Let $\phi(\cdot)$ be a nonconstant, bounded, continuous and monotone increasing function and $\epsilon > 0$. Then, given any continuous real function $f$ on a compact subset of $R^n$ such as $[0,1]^n$, $\exists$ vectors $\ve{w}_1, \ve{w}_2,...,\ve{w}_n, \ve{\alpha}$ and $\ve{b}$ and a parametrized function $G(\cdot,\ve{w},\ve{\alpha},\ve{b}):[0,1]^n \to \mathbb{R}$ such that:
\begin{displaymath}
\vert G(\ve{x},\ve{w},\ve{\alpha},\ve{b}) - f(x)\vert < \epsilon \qquad \forall \ve{x} \in [0,1]^n
\end{displaymath} (13)


\begin{displaymath}
G(\ve{x},\ve{w},\ve{\alpha},\ve{b}) = \sum_{j=1}^N \alpha_j \varphi(\ve{w}_j^t \ve{x}+b_j)
\end{displaymath} (14)

$\ve{w}_j \in \mathbb{R}^n,\alpha_j,b_j \in \mathbb{R}, \ve{b} = (b_1,b_2,...,b_N)$



Pedro Larroy 2005-04-29