Self-Organizing Systems, 1963

Part 1

Transcriber’s Notes:

Underscores “” before and after a word or phrase indicate italics_ in the original text. Equal signs “=” before and after a word or phrase indicate =bold= in the original text. A single underscore after a symbol indicates a subscript. Carat symbol “^” designates a superscript. A bold arrow in front of a letter indicates a vector, e.g. ⮕M means “the vector M”. Small capitals have been converted to SOLID capitals. Illustrations have been moved so they do not break up paragraphs. Typographical and punctuation errors have been silently corrected.

SELF-ORGANIZING SYSTEMS 1963

=Edited By=

=JAMES EMMETT GARVEY= Office of Naval Research Pasadena, California

=ACR-96=

=OFFICE OF NAVAL RESEARCH DEPARTMENT OF THE NAVY WASHINGTON, D.C.=

For sale by the Superintendent of Documents. U.S. Government Printing Office Washington, D.C., 20402—Price $1.50

CONTENTS

Foreword iv

The Ionic Hypothesis and Neuron Models 1 —E. R. Lewis

Fields and Waves in Excitable Cellular Structures 19 —R. M. Stewart

Multi-Layer Learning Networks 37 —R. A. Stafford

Adaptive Detection of Unknown Binary Waveforms 46 —J. J. Spilker, Jr.

Conceptual Design of Self-Organizing Machines 52 —P. A. Kleyn

A Topological Foundation for Self-Organization 65 —R. I. Ścibor-Marchocki

On Functional Neuron Modeling 71 —C. E. Hendrix

Selection of Parameters for Neural Net Simulations 76 —R. K. Overton

Index of Invited Participants 77

FOREWORD

The papers appearing in this volume were presented at a Symposium on Self-Organizing Systems, which was sponsored by the Office of Naval Research and held at the California Institute of Technology, Pasadena, California, on 14 November 1963. The Symposium was organized with the aim of providing a critical forum for the presentation and discussion of contemporary significant research efforts, with the emphasis on relatively uncommon approaches and methods in an early state of development. This aim and nature dictated that the Symposium be in effect a Working Group, with numerically limited invitational participation.

The papers which were presented and discussed did in fact serve to introduce several relatively unknown approaches; some of the speakers were promising young scientists, others had become known for contributions in different fields and were as yet unrecognized for their recent work in self-organization. In addition, the papers as a collection provided a particularly broad, cross-disciplinary spectrum of investigations which possessed intrinsic value as a portrayal of the bases upon which this new discipline rests. Accordingly, it became obvious in retrospect that the information presented and discussed at the Symposium was of considerable interest—and should thus receive commensurate dissemination—to a much broader group of scientists and engineers than those who were able to participate directly in the meeting itself. This volume is the result of that observation; as an edited collection of the papers presented at the Symposium, it forms the Proceedings thereof. If it provides a useful reference for present and future investigators, as well as documenting the source of several new approaches, it will have fulfilled its intended purpose well.

A Symposium which takes the nature of a Working Group depends for its utility especially upon effective commentary and critical analysis, and we commend all the participants for their contributions in this regard. It is appropriate, further, to acknowledge the contributions to the success of the Symposium made by the following: The California Institute of Technology for volunteering to act as host and for numerous supporting services; Professor Gilbert D. McCann, Director of the Willis Booth Computing Center at the California Institute of Technology, and the members of the technical and secretarial staffs of the Computing Center, who assumed the responsibility of acting as the immediate representatives of the Institute; the members of the Program Committee, who organized and led the separate sessions—Harold Hamilton of General Precision, Joseph Hawkins of Ford Motor Company, Robert Stewart of Space-General, Peter Kleyn of Northrop, and Professor McCann; members of the Technical Information Division of the Naval Research Laboratory, who published these Proceedings; and especially the authors of the papers, which comprised the heart of the Symposium and subsequently formed this volume. To all of these the sponsors wish to express their very sincere appreciation.

JAMES EMMETT GARVEY Office of Naval Research Branch Office Pasadena, California

MARGO A. SASS Office of Naval Research Washington, D.C.

The Ionic Hypothesis and Neuron Models

E. R. LEWIS

Librascope Group, General Precision, Inc. Research and Systems Center Glendale, California

Part 2

The measurements of Hodgkin and Huxley were aimed at revealing the mechanism of generation and propagation of the all-or-none spike. Their results led to the Modern Ionic Hypothesis. Since the publication of their papers in 1952, advanced techniques with microelectrodes have led to the discovery of many modes of subthreshold activity not only in the axon but also in the somata and dendrites of neurons. This activity includes synaptic potentials, local response potentials, and pacemaker potentials.

We considered the question, “Can this activity also be explained in terms of the Hodgkin-Huxley Model?” To seek an answer, we have constructed an electronic analog based on the ionic hypothesis and designed around the data of Hodgkin and Huxley. Synaptic inputs were simulated by simple first-order or second-order networks connected directly to simulated conductances (potassium or sodium). The analog has, with slight parameter adjustments, produced all modes of threshold and subthreshold activity.

INTRODUCTION

In recent years physiologists have become quite adept at probing into neurons with intracellular microelectrodes. They are now able, in fact, to measure (a) the voltage change across the postsynaptic membrane elicited by a single presynaptic impulse (see, for examples, references 1 and 2) and (b) the voltage-current characteristics across a localized region of the nerve cell membrane (3), (4), (5), (6). With microelectrodes, physiologists have been able to examine not only the all-or-none spike generating and propagating properties of axons but also the electrical properties of somatic and dendritic structures in individual neurons. The resulting observations have led many physiologists to believe that the individual nerve cell is a potentially complex information-processing system far removed from the simple two-state device envisioned by many early modelers. This new concept of the neuron is well summarized by Bullock in his 1959 Science article (10). In the light of recent physiological literature, one cannot justifiably omit the diverse forms of somatic and dendritic behavior when assessing the information-processing capabilities of single neurons. This is true regardless of the means of assessment—whether one uses mathematical idealizations, electrochemical models, or electronic analogs. We have been interested specifically in electronic analogs of the neuron; and in view of the widely diversified behavior which we must simulate, our first goal has been to find a unifying concept about which to design our analogs. We believe we have found such a concept in the Modern Ionic Hypothesis, and in this paper we will discuss an electronic analog of the neuron which was based on this hypothesis and which simulated not only the properties of the axon but also the various subthreshold properties of the somata and dendrites of neurons.

We begin with a brief summary of the various types of subthreshold activity which have been observed in the somatic and dendritic structures of neurons. This is followed by a brief discussion of the Hodgkin-Huxley data and of the Modern Ionic Hypothesis. An electronic analog based on the Hodgkin-Huxley data is then introduced, and we show how this analog can be used to provide all of the various types of somatic and dendritic activity.

SUBTHRESHOLD ELECTRICAL ACTIVITY IN NEURONS

In studying the recent literature in neurophysiology, one is immediately struck by the diversity in form of both elicited and spontaneous electrical activity in the single nerve cell. This applies not only to the temporal patterns of all-or-none action potentials but also to the graded somatic and dendritic potentials. The synaptic membrane of a neuron, for example, is often found to be electrically inexcitable and thus incapable of producing an action potential; yet the graded, synaptically induced potentials show an amazing diversity in form. In response to a presynaptic impulse, the postsynaptic membrane may become hyperpolarized (inhibitory postsynaptic potential), depolarized (excitatory postsynaptic potential), or remain at the resting potential but with an increased permeability to certain ions (a form of inhibition). The form of the postsynaptic potential in response to an isolated presynaptic spike may vary from synapse to synapse in several ways, as shown in Figure 1. Following a presynaptic spike, the postsynaptic potential typically rises with some delay to a peak value and then falls back toward the equilibrium or resting potential. Three potentially important factors are the delay time (synaptic delay), the peak amplitude (spatial weighting of synapse), and the rate of fall toward the equilibrium potential (temporal weighting of synapse). The responses of a synapse to individual spikes in a volley may be progressively enhanced (facilitation), diminished (antifacilitation), or neither (1), (2), (7), (8). Facilitation may be in the form of progressively increased peak amplitude, or in the form of progressively decreased rate of fall (see Figure 2). The time course and magnitude of facilitation or antifacilitation may very well be important synaptic parameters. In addition, the postsynaptic membrane sometimes exhibits excitatory or inhibitory aftereffects (or both) on cessation of a volley of presynaptic spikes (2), (7); and the time course and magnitude of the aftereffects may be important parameters. Clearly, even if one considers the synaptic potentials alone, he is faced with an impressive variety of responses. Examples of the various types of postsynaptic responses may be found in the literature, but for purposes of the present discussion the idealized wave forms in Figure 2 will demonstrate the diversity of electrical behavior with which one is faced.

Continue reading Self-Organizing Systems, 1963 free in the Wunder reader →