| |||||
| | |||||
One way to provide additional constraints is through the use of "hints" [361], [416]. In addition to outputs for the function of interest, extra output nodes are added to the network and trained to learn certain hint functions. The hint functions should be related to the function of interest and are usually designed to be easier to learn. The extra functions may speed convergence by generating nonzero derivatives in regions where the original function has plateaued. They may also aid generalization by providing additional constraints and removing certain local minima of the original function. They discourage the choice of a solution that somehow matches the original function on the training samples but does not include intermediate concepts embedded in the hints. After training, the hint output nodes can be removed because they usually are not of interest in the overall system.
The term hints is usually used to refer to augmented outputs, but hint information can also be provided in the form of targets for the (normally) hidden nodes. Hints can also be provided by shaping the target function dynamically [193]. The initial target function is an easy to learn, coarse approximation of the desired function which is gradually made more similar to the desired function as the learner masters each stage. This is a standard technique in animal training.
