| |||||
| | |||||
The methods considered so far are mostly numerical techniques that make no use of problem-specific information. Another powerful way of favoring good generalization is through the use of domain-dependent prior information.
As noted earlier, samples alone are not enough to uniquely specify the target function in the absence of other constraints. In many applications where neural nets are considered, there is significant human knowledge that could be useful even though it is incomplete or only partially reliable. There may be existing techniques that give reasonable but imperfect solutions or we may know certain rules that should be satisfied by any correct solution. When the goal is to develop a working application, it makes sense to use as much of this information as possible.
The following sections review some ways of using domain-dependent prior information in a neural network. Some are based on the idea of adapting a good non-neural solution to provide the starting point for further fine tuning in a neural network structure. It should be noted that whether or not this leads to good generalization depends on many factors;in some cases it may merely accelerate learning by giving the network a good headstart, without really improving generalization.
