Neural Network Structures

Figure above is a simple mathematical model for a neuron. The unit's output activation is
, where ai is the output activation of unit i and w_ij is the weight on the link from unit i to this unit.
Neural Networks are composed of nodes or units cormected by directed links. A link from unit i to unit j serves to propagate the activation a_i from i to j. Each link also has a numeric weight w_i,j associated with it, which determines the strength and sign of the connection. Just as in linear regression models, each unit has a dummy input a₀= 1 with an associated weight w_0,j.
Each unit j first computes a weighted sum of its inputs:

Then it applies an activation function g to this sum to derive the output:


Having decided on the mathematical model for individual "neurons”, the next task is to connect them together to form a network. There are two fundamentally distinct ways to do this.

• A recurrent network, on the other hand, feeds its outputs back into its own inputs. This means that the activation levels of the network form a dynamical system that may reach a stable state or exhibit oscillations or even chaotic behavior. Moreover, the response of the network to a given input depends on its initial state, which may depend on previous inputs. Hence, recurrent networks (unlike feed-forward networks) can support short-term memory.
• A feed-forward network, has connections only in one direction-that is, it forms a directed acyclic graph. Every node receives input from "upstream" nodes and delivers output to "downstream" nodes; there are no loops. A feed-forward network represents a function of its current input; thus, it has no internal state other than the weights themselves.

Feed-forward networks are usually arranged in layers, such that each unit receives input only from units in the immediately preceding layer.

• single-layer networks, in which every unit connects directly from the network's inputs to its outputs, and
• multilayer networks, which have one or more layers of hidden units that are not connected to the outputs of the network.

Related Posts :
Neural Network Classification
Neural Networks Single Layer
Neural Networks Multi Layer
Data Classification