Part 10

The threshold value, Γᵢ, is adapted so that it increases if the memory becomes a better replica of the signal with the result that γᵢ increases. On the other hand, if the memory is a poor replica of the signal (for example, if it contains noise alone), it is necessary that the threshold decay with time to the point where additional acceptances can modify the memory structure.

The experimental machine is entirely digital in operation and, as stated above, is capable of recovering waveforms of up to 10³ samples in duration. In a typical experiment, one might attempt to recover an unknown noise-perturbed, pseudo-random waveform of up to 10³ bits duration which occurs at random intervals. If no information is available as to the signal waveshape, the adaptive memory is blank at the start of the experiment.

In order to illustrate the operation of the machine most clearly, let us consider a repetitive binary waveform which is composed of 10³ bits of alternate “zeros” and “ones.” A portion of this waveform is shown in Figure 2a. The waveform actually observed is a noise-perturbed version of this waveform shown in Figure 2b at-6 db signal-to-noise ratio. The exact sign of each of the signal bits obviously could not be accurately determined by direct observation of Figure 2b.

Figure 2—Binary signal with additive noise at-6 db SNR]

Figure 3—Adaption of the memory at-6 db SNR: (a) Blank initial memory; (b) Memory after first dump; (c) Memory after 12 dumps; (d) Memory after 40 dumps; (e) Perfect “checkerboard” memory for comparison]

As the machine memory adapts to this noisy input signal, it progresses as shown in Figure 3. The sign of 10^{3} memory components are displayed in a raster pattern in this figure. Figure 3a shows the memory in its blank initial state at the start of the adaption process. Figure 3b shows the memory after the first adaption of the memory. This first “dump” occurred after the threshold had decayed to the point where an energy measurement produced an acceptance decision. Figure 3c and 3d show the memory after 12 and 40 adaptions, respectively. These dumps, of course, are based on both energy and cross-correlation measurements. As can be seen, the adapted memory after 40 dumps is already quite close to the perfect memory shown by the “checkerboard” pattern of Figure 3c.

The detailed analysis of the performance of this type of machine vs. signal-to-noise ratio, average signal repetition rate, signal duration, and machine parameters is extremely complex. Therefore, it is not appropriate here to detail the results of the analytical and experimental work on the performance of this machine. However, several conclusions of a general nature can be stated.

(a) Because the machine memory is always adapting, there is a relatively high penalty for “false alarms.” False alarms can destroy a perfect memory. Hence, the threshold level needs to be set appropriately high for the memory adaption. If one wishes to detect signal occurrences with more tolerance to false alarms, a separate comparator and threshold level should be used.

(b) The present machine structure, which allows for slowly varying changes in the signal waveshape, exhibits a marked threshold effect in steady-state performance at an input signal-to-noise ratio (peak signal power-to-average noise power ratio) of about -12 db. Below this signal level, the time required for convergence increases very rapidly with decreasing signal level. At higher SNR, convergence to noise-like signals, having good auto-correlation properties, occurs at a satisfactory rate.

A more detailed discussion of performance has been published in the report cited in footnote reference 1.

Conceptual Design of Self-Organizing Machines

P. A. KLEYN

Northrop Nortronics Systems Support Department Anaheim, California

Self-organization is defined and several examples which motivate this definition are presented. The significance of this definition is explored by comparison with the metrization problem discussed in the companion paper (1) and it is seen that self-organization requires decomposing the space representing the environment. In the absence of a priori knowledge of the environment, the self-organizing machine must resort to a sequence of projections on unit spheres to effect this decomposition. Such a sequence of projections can be provided by repeated use of a nilpotent projection operator (NPO). An analog computer mechanization of one such NPO is discussed and the signal processing behavior of the NPO is presented in detail using the Euclidean geometrical representation of the metrizable topology provided in the companion paper. Self-organizing systems using multiple NPO’s are discussed and current areas of research are identified.

INTRODUCTION

Unlike the companion paper which considers certain questions in depth, this paper presents a survey of the scope of our work in self-organizing systems and is not intended to be profound.

The approach we have followed may be called phenomenological (Figure 1). That is, the desired behavior (self-organization) was defined, represented mathematically, and a mechanism(s) required to yield the postulated behavior was synthesized using mathematical techniques. One advantage of this approach is that it avoids assumptions of uniqueness of the mechanism. Another advantage is that the desired behavior, which is after all the principal objective, is taken as invariant. An obvious disadvantage is the requirement for the aforementioned synthesis technique; fortunately in our case a sufficiently general technique had been developed by the author of the companion paper.

From the foregoing and from the definition of self-organization we employ (see conceptual model), it would appear that our research does not fit comfortably within any of the well publicized approaches to self-organization (2). Philosophically, we lean toward viewpoints expressed by Ashby (3), (4), Hawkins (5), and Mesarovic (6) but with certain reservations. We have avoided the neural net approach partly because it is receiving considerable attention and also because the brain mechanism need not be the unique way to produce the desired behavior.

Nor have we followed the probability computer or statistical decision theory approach exemplified by Braverman (7) because these usually require some sort of preassigned coordinate system (8). Neither will the reader find much indication of formal logic (9) or heuristic (10) programming. Instead, we view a self-organizing system more as a mirror whose appearance reflects the environment rather than its own intrinsic nature. With this viewpoint, a self-organizing system appears very flexible because it possesses few internal constraints which would tend to distort the reflection of the environment and hinder its ability to adapt.

CONCEPTUAL MODEL

Definition

A system is said to be self-organizing if, after observing the input and output of an unknown phenomenon (transfer relation), the system organizes itself into a simulation of the unknown phenomenon.

Implicit in this definition is the requirement that the self-organizing machine (SOM) not possess a preassigned coordinate system. In fact it is just this ability to acquire that coordinate system implicit in the input-output spaces which define the phenomenon that we designate as self-organization. Thus any a priori information programmed into the SOM by means of, for example, stored or wired programs, constrains the SOM and limits its ability to adapt. We do not mean to suggest that such preprogramming is not useful or desirable; merely that it is inconsistent with the requirement for self-organization. As shown in Figure 2, it is the given portion of the environment which the SOM is to simulate, which via the defining end spaces, furnishes the SOM with all the data it needs to construct the coordinate system intrinsic to those spaces.

The motivation for requiring the ability to simulate as a feature of self-organization stems from the following examples.

Consider the operation of driving an automobile. Figure 3 depicts the relation characterized by a set of inputs; steering, throttle, brakes, transmission, and a set of outputs; the trajectory. Operation of the automobile requires a device (SOM) which for a desired trajectory can furnish those inputs which realize the desired trajectory. In order to provide the proper inputs to the automobile, the SOM must contain a simulation of ⨍⁻¹(x).

Since ⨍(x) is completely defined in terms of the inputs and the resulting trajectories, exposure to them provide the SOM with all the information necessary to simulate ⨍⁻¹(x). And if the SOM possesses internal processes which cause rearrangement of the input-output relation of the SOM to correspond to ⨍⁻¹(x) in accordance with the observed data, the SOM can operate an automobile. It is this internal change which is implied by the term “self-organizing,” but note that the instructions which specify the desired organization have their source in the environment.

As a second example consider adaptation to the environment. Adapt (from Webster) means: “to change (oneself) so that one’s behavior, attitudes, etc., will conform to new or changed circumstances. Adaptation in biology means a change in structure, function or form that produces better adjustment to the environment.” These statements suggest a simulation because adjustment to the environment implies survival by exposing the organism to the beneficial rather than the inimical effects of the environment. If we represent the environment (or portion thereof) as a relation as shown in Figure 2, we note that the ability to predict what effect a given disturbance will have is due to a simulation of the cause-effect relation which characterizes the environment.

It would be a mistake to infer from these examples that simulation preserves the appearance of the causes and effects which characterize a relation. We clarify this situation by examining a relation and its simulation.