It is proposed that all weights in elements attached to inputs which
come from other elements in the network be restricted to positive
values. (Weights attached to the original inputs to the network, of
course, must be allowed to be of either sign.) The reason for such a
restriction is this. If element 1 is an input to element 2 with weight
c₁₂, element 2 to element 3 with weight c₂₃, _etc._, then the sign of
the product, c₁₂c₂₃ ..., gives the sense of the effect of a change in
the output of element 1 on the final element in the chain (assuming
this is the only such chain between the two elements). If these various
weights were of either possible sign, then a decision as to whether or
not to change the output in element 1 to help correct an error in the
final element would involve all weights in the chain. Moreover, since
there would in general be a multiplicity of such chains, the decision
is rendered impossibly difficult.
The above restriction removes this difficulty. If the output of any
element in the network is changed, say, from -1 to +1, the effect on
the final element, if it is affected at all, is in the same direction.
It should be noted that this restriction does not seriously affect
the logical capabilities of a network. In fact, if a certain logical
function can be achieved in a network with the use of weights of
unrestricted sign, then the same function can be generated in another
network with only positive interconnecting weights and, at worst, twice
the number of elements. In the worst case this is done by generating
in the restricted network both the output and its complement for each
element of the unrestricted network. (It is assumed that there are no
loops in the network.)