16.11
Knowledge-Based Neural Nets
Rule-based systems, such as expert systems, have been used
quite successfully in many applications. These systems use human information
efficiently and there is interest in developing hybrid systems combining the
high-level information processing abilities of symbolic systems with the
adaptability of neural nets. A useful feature of expert systems which neural
networks generally lack is the ability to explain the reasoning behind its
conclusions.
One approach [342], [375] is to embed symbolic rules in the initial structure of a
neural network by translating the AND, OR, and NOT terms into corresponding
network structures with appropriate weights. (Simple variable-free propositional
rules are easily translated to neural network structures.) Additional links with
small random weights are provided to let the system add other terms that may be
useful. The network is then trained from examples to improve its performance.
Because the embedded symbolic rules are often classifications, the cross-entropy
error function may work better than the mean-squared-error function [342].
Besides faster training due to a good initial solution,
improved generalization has been observed in spite of imperfect embedded rules.
This is attributed to "(1) focusing attention on relevant input features, and
(2) indicating useful intermediate conclusions (which suggest a good network
topology)" [342]. Given a sufficient number of examples, a standard
network initialized with random weights should converge to the same asymptotic
performance, but the knowledge-based networks generalize better when examples
are sparse. Evidently "the initial knowledge is ‘worth’ some number of training
examples" [342]. Some references for ways of using forms of prior
knowledge other than symbolic rules are provided by Shavlik [342].