Softmax function

The final output is calculated by a softmax function, controlled by the gain parameter G, over all the units in the output layer ... 

The network consists of two input neurons for presentation of the two v-values as well as four output neurons, y, which represent the four classes (cf Figure 8.15). In addition, a hidden layer was added with up to 20 neurons and the intercepts of the surfaces are modeled by bias neurons to both the hidden and output layers. The transfer function in the neurons of the hidden layer was of sigmoid type, and aggregation of the neurons in the output layer was carried out by calculating the normalized exponentials (softmax criterion). 

By assigning a softmax activation function on the output layer of the neural network (or a softmax component in a component-based neural network) for categorical target variables, the outputs can be interpreted as posterior probabilities. This is very useful in classification as it gives a certainty measure on classifications. 

---