It would appear from some of the preliminary results of network
simulations that it may be useful to have elements become “fatigued”
after undergoing an excessive number of weight changes. Experiments
have been performed on simplifications of the model described so far
which had the occasional result that a small number of elements came
to a state where they received most of the weight increments, much
to the detriment of the learning process. In such cases the network
behaves as if it were composed of many fewer adjustable elements. In a
sense this is asking each element to maintain a record of the data it
is being asked to store so that it does not attempt to exceed its own
information capacity.
It is not certain just how this fatigue factor should enter in the
element’s actions, but if it is to be compatible with the variable bias
method, this fatigue factor must enter into the element’s response to
a changing bias. Once an element changes state with zero sum at the
same time that the final output becomes wrong, incrementing must occur
if the method is to work. Hence a “fatigued” element must respond less
energetically to a change of bias, perhaps with a kind of variable
factor to be multiplied by the bias term.