Learning of neural network takes place on the basis of a sample of the population under study. During the course of learning, compare the value delivered by output unit with actual value. After that adjust the weights of all units so to improve the prediction.
There are many Neural Network Algorithms are available for training Artificial Neural Network. Let us now see some important Algorithms for training Neural Networks:
We use the gradient descent algorithm to find the local smallest of a function. This algorithm converges to the local smallest. By approaching proportional to the negative of the gradient of the function. To find local maxima, take the steps proportional to the positive gradient of the function. This is gradient ascendant process.
In linear models, error surface is well defined and well known mathematical object in shape of a parabola. Then find the least point by calculation. Unlike linear models, neural networks are complex nonlinear models. Here, the error surface has an irregular layout, crisscrossed with hills, valleys, plateau, and deep ravines. To find the least point on this surface, for which no maps are available, the user must explore it.
In this algorithm, you move over the error surface by following the line with the greatest slope. It also offers the possibility of reaching the lowest possible point. You then have to work out at the optimal rate at which you should travel down the slope.
The correct speed is proportional to the slope of the surface and the learning rate. Learning rate controls the extent of modification of the weights during the learning process.
Hence, the moment of a neural network can affect the performance of multilayer perceptron.
Source for picture: click here
This algorithm based on the concept of natural selection or survival of the fittest in Biology. Concept of natural selection states that – for a given population, environment conditions use a pressure that results in the rise of the fittest in that population.
To measure fittest in a given population, you can apply a function as an abstract measure.
In the context of evolutionary algorithms, refer recombination to as an operator. Then apply it to two or more candidates known as parents, and result in one of more new candidates known as children. Apply the mutation on a single candidate and results in a new candidate. By applying recombination and mutation, we can get a set of new candidates to place in the next generation based on their fittest measure.
The two basic elements of evolutionary algorithms are:
The common features of evolutionary algorithms are:
Hence, on the basis of details and applied problems, we use various formats of evolutionary algorithms.
Some common evolutionary algorithms are:
In all these algorithms, genetic algorithm is the most common evolutionary algorithm.
Genetic algorithms, developed by John Holland’s group from the early 1970s. It enables the most appropriate rules for the solution of a problem to be selected. So that they send their ‘genetic material’ (their variables and categories) to ‘child’ rules.
Here refer a as a set of categories of variables. For example, customers aged between 36 and 50, having financial assets of less than $20,000 and a monthly income of more than $2000.
A rule is the equal of a branch of a decision tree; it is also analogous to a gene. You can understand genes as units inside cells that control how living organisms inherit features of their parents. Thus, Genetic algorithms aim to reproduce the mechanisms of natural selection. By selecting the rules best adapted to prediction and by crossing and mutating them until get a predictive model.
Together with neural networks, they form the second type of algorithm. Which mimics natural mechanisms to explain phenomena that are not necassary natural.
The steps for executing genetic algorithms are:
Crossing of 2 rules is exchange of some of their variables or categories to produce 2 new rules. Crossing is more common than mutation.
Algorithm ends when 1 of the following 2 conditions meets:
In Conclusion, Artificial Neural Network is typically difficult to configure and slow to train, but once prepared are very fast in the application. They are generally designed as models to overcome the mathematical, computational, and engineering problems. Since, there is a lot of research in mathematics, neurobiology and computer science.