Basic Convolutional Neural Network tutorial

Neural Network

A Neural Network is a feed forward neural network with one or more hidden layers. The network consists of an input layer of source neurons, at least one middle or hidden layer of computational neurons, and an output layer of computational neurons. The input signals are propagated in a forward direction on a layer-by-layer basis.


Three-layer back-propagation neural network

Summary of Back Propagation learning algorithm

Set learning rate
Set initial weight values (incl. biases):  w, v
Loop until stopping criteria satisfied:

    present input pattern to input units
    compute functional signal for hidden units
    compute functional signal for output units

   present Target response to output units
   computer error signal for output units
   compute error signal for hidden units
   update all weights at same time
   increment n  to n+1 and select next input and target
end loop

Convolutional Neural Network

Convolutional Neural Network(CNN) are trainable multistage
architecture. CNN implements the three architectural ideas to
ensure some degree of shift , scale, and distortion invarience, when
fully connected networks are used.

1 Local receptive fields : by this it takes into consideration topology of input to
   calculate features.

2 Shared Weights : by this it can learn feature using less number of trainable
    parameters compared to fully connected neural networks.

3 Spatial or Temporal Subsampling : to achive shifts and distortion invariance.