Learning model

Learning models:

• Error correction
• Minimum distance
• Hebbian learning (correlational synapse):
  • If two neurons in either side of a synapse are activated simultaneously, the strength of that synapse is selectively increased.
  • If two neurons in either side of a synapse are activated asynchronously, then that synapse is weakened.
  • Hebb's:
    
    $$\Delta w_{kj} = \eta y_k(n) x_j(n)$$
    
    

  • Covariance¹:
    
    $$\Delta w_{kj} = \eta (x_j - m_x)(y_k - m_y)$$
    
    
    • Weight $w_kj$ is increased if $x_j > m_x$ and $y_k > m_y$
    • Weight $w_kj$ is decreased if $x_j > m_x$ and $y_k < m_y$
    • Weight $w_kj$ is decreased if $x_j < m_x$ and $y_k > m_y$
• Competitive learning
  A set of neurons compete to become active.
  • Start with equal neurons with some random variations of weights
  • Only one neuron per group is active at at time (winner-takes-all)
  • Neuron that wins relinquishes weight that is distributed along the active input nodes
    
    $$\Delta w_{kj} = \left\lbrace \begin{array}{ll} \eta (x_j ... ...ins} \\ 0 & \textrm{if neuron k loses} \end{array} \right.$$
    
    

  $\sum_{j} w_{kj}^2 = 1 \quad \forall \quad k$
• Boltzmann learning
• Supervised learning: error correction.
• Unsupervised learning: without reference or desired response.