A statistical approach to the theory of the central nervous system

A “probabilistic” rather than a “deterministic” approach to the theory of neural nets is developed. Neural nets are characterized by certain parameters which give the probability distributions of different kinds of synaptic connections throughout the net. Given a “state” of the net (i.e., the distribution of firing neurons) at a given moment, an equation for the state at the next moment of quantized time is deduced. Certain very special cases involving constant distributions are solved. A necessary condition for a steady state is deduced in terms of an integral equation, in general non-linear.     

Responses of recurrent nets of asymmetric ON and OFF cells

A neural field model of ON and OFF cells with all-to-all inhibitory feedback is investigated. External spatiotemporal stimuli drive the ON and OFF cells with, respectively, direct and inverted polarity. The dynamic differences between networks built of ON and OFF cells (“ON/OFF”) and those having only ON cells (“ON/ON”) are described for the general case where ON and OFF cells can have different spontaneous firing rates; this asymmetric case is generic. Neural responses to nonhomogeneous static and time-periodic inputs are analyzed in regimes close to and away from self-oscillation. Static stimuli can cause oscillatory behavior for certain asymmetry levels. Time-periodic stimuli expose dynamical differences between ON/OFF and ON/ON nets. Outside the stimulated region, we show that ON/OFF nets exhibit frequency doubling, while ON/ON nets cannot. On the other hand, ON/ON networks show antiphase responses between stimulated and unstimulated regions, an effect that does not rely on specific receptive field circuitry. An analysis of the resonance properties of both net types reveals that ON/OFF nets exhibit larger response amplitude. Numerical simulations of the neural field models agree with theoretical predictions for localized static and time-periodic forcing. This is also the case for simulations of a network of noisy integrate-and-fire neurons. We finally discuss the application of the model to the electrosensory system and to frequency-doubling effects in retina.     

Application of information networks to a theory of vision

Some principles of information theory are utilized in the design of neural nets of the McCulloch-Pitts type. In particular, problems are considered where signals from several neurons must pass through a single one, thus resulting in a “bottleneck” in the flow of information, an abstract model of the corresponding bottleneck from the retina to the optic nerve. The first part of the paper deals with a construction of a McCulloch-Pitts net in which the redundancy in the messages originating in two neurons is utilized so that the messages can be sent over a single neuron with little loss of information. In the second part, messages from a set of neurons are “pumped” into two channel neurons. The optimum connection scheme is computed for this case, i.e, one resulting in a minimum loss of information. Possible biological implications of this approach are indicated.     

A new model for simple neural nets and its application in the design of a neural oscillator

The results of a previous theoretical study of a class of systems are applied for the design of neural nets which try to simulate biological behavior. Besides the models for single aperiodic and periodic neurons, a “neural oscillator” is developed which consists of two cross-excited neurons. Its response is similar to the firing pattern of certain biological neural oscillators, like the flying system of the locust. Also, by proper change of its parameters, it can be made highly irregular, providing a deterministic model for the spontaneous neural activity.     

A note on the quantum-theoretic basis of primary genetic activity

In previous work (Bull. Math. Biophysics,23, 393–403, 1960) it was shown that, if primary genetic processes are of an essentially microphysical nature, the objects bearing the primary genetic information must act in a catalytic fashion. At the same time it was pointed out that the kind of catalysis involved in the primary genetic process was fundamentally different, in specific ways, from that occurring, e.g., in enzyme systems. The present work demonstrates that, if the information-bearing objects of the general theory are identified with molecules of DNA, and the primary gene products are considered to be RNA of the “messenger” variety, then the predictions of the general theory can be compared with experimental data from various recently isolated polymerase systems, which appear to “copy” a sequence of nucleotides from DNA into RNAin vitro, and with certainin vivo microbial systems. It is found that these data provide detailed support for the conclusions drawn from the general theory. However, it is emphasized that the identification of the information-bearing objects and primary gene products as DNA and RNA respectively, which allows us to compare the theory with the cited data, is by no means the only identification which can be made; i.e., other interpretations of the general theory are certainly not precluded. This research was supported by the United States Air Force through the Air Force Office of Scientific Research of the Air Research and Development Command, under Grant No. AF-AFOSR-9-63.     

On the reversibility of environmentally induced alterations in abstract biological systems

The environmentally induced alterations in structure of (M, ℜ) which were described previously (R. Rosen,Bull. Math. Biophysics,23, 165–171, 1961) are examined from the standpoint of determining under what circumstances they can be reversed by further environmental interactions. For simplicity we consider only the case of (M, ℜ)-systems possessing one “metabolic” and one “genetic” component. In the case of environmentally induced alteration of the “metabolic” component alone, a necessary and sufficient condition is given for the reversibility of the alteration. In the case of alteration of the “genetic” component, the situation becomes more complex; several partial results are given, but a full analysis is not available at this time. Some possible biological implications of this analysis are discussed. This research was supported by the United States Air Force through the Air Force Office of Scientific Research of the Air Research and Development Command, under Contract no. AF-49(638)-917 and Grant no. AF-AFOSR-9-63.     

Interaction of cycles

We discuss under the McCulloch and Pitts assumptions for neural nets a circuit consisting ofk cycles such that one cycle is activated by an outside stimulus and sends an impulse to a second cycle which in its turn sends an impulse to the next cycle, etc., up to thekth cycle, which sends an impulse to a response. We thus have a “series” ofk cycles “interacting”. We give several theorems regarding the response patterns of such circuits under the additional constraint that the stimulus acts but once, and at the time it acts the circuit is at rest.     

An exact method for the computation of the connectivity of random nets

The problem of finding the “weak connectivity” of a random net is reduced to one involving a Markov process. This provides a mathematically exact treatment of the problem which had previously been treated by an approximation, whose justification was not rigorous. The exact method allows in principle not only the calculation of the “weak connectivity”, but also of the “strong connectivity”, and, in general, the probability that from a randomly selected neuron in the net there exist paths to a specified number of neurons. The computations become exceedingly involved for large nets.     

The smallest value of the axon density for which ‘ignition’ can occur in a random net

As shown by A. Rapoport (1952), when a very brief stimulation or “instantaneous input” is applied to a random net, the subsequent events are determined by the parameters of the net as follows: If the axon densitya is sufficiently large and the fraction γ of the neurons initially stimulated exceeds a certain value γ₁ (theover-all threshold of the net for instantaneous stimulation), excitation will spread through the net until a steady state is reached in which a fraction γ₂ ⩾ γ₁ of the neurons is firing (“ignition phenomenon”). If γ < γ₁ the activity in the net dies out. However, if the axon density is too small, the activity will ultimately die out, no matter how large the fraction of initially stimulated neurons. Thus there exists a limiting valueA of the axon density below which the net cannot “ignite”. ThisA is a function ofh, theindividual threshold of the neurons constituting the net (we assume hereh≥2, since forh=1 the situation is essentially different). Geometrically γ₁ and γ₂ are determined as the two intersection points of a straight line with a sigmoid curve. Whena<A the two curves do not intersect and fora=A they are tangent. In this paper the “tangency case” is investigated and the general features of the functionA(h) are determined. It is shown thatA increases monotonically withh (as one would expect). For all values ofh>1 we haveA(h)>h, but the fractionA(h)/h and the derivativedA(h)/dh approach unity ash increases. An analytical expression of the functionA(h) valid for very large values ofh is derived.     

A logical calculus of the ideas immanent in nervous activity

Because of the “all-or-none” character of nervous activity, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiological assumptions are equivalent, in the sense that for every net behaving under one assumption, there exists another net which behaves under the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed. Reprinted from theBulletin of Mathematical Biophysics, Vol. 5, pp. 115–133 (1943).     

Beyond the binary case in random nets

Experiments on random binary, ternary, etc. (P=2, 3,…, 10) switching nets are reported. Behavioral cycle lengths are examined as functions of output variety,P, input connectance,K, and net size,N. Overall, output variety appears an influential, well-behaved net property. Strong, but well-behaved interactions appear among net variables. In high connectance nets, median cycle length grows approx. asP ^N/2. Other factors constant, one-connected nets show the shortest cycles, and connectance effects appear to converge asymptotically aroundN. Data for cycle length as a function of net size suggest a concavity not compatible with the Kauffman “square root law” (Kauffman, 1969). Evidence of a positive relationship between cycle length and run-in length is found in two-input nets; weaker evidence is obtained that in higher connectance nets this relationship becomes negative in sign. The “modular complexity” ofP>2 nets is examined briefly.     

Response time and threshold of a random net

The response time of a random net is defined as the expected time (measured in the number of synaptic delays) required for the excitation in the net (measured by the fraction of neurons firing per unit time) to reach a certain level. The response time is calculated in terms of the net parameters as a function of the intensity of the outside stimulation. Two principal types of cases are studied, 1) an instantaneous initial stimulation, and 2) continuously applied stimulation. It is shown that for a certain type of net where the required level of excitation is small, the response time-intensity equation reduces to the one derived on the basis of the “one-factor” theory applied to a neural connection. More general assumptions, however, give different types of equations. The concept of the “net threshold” is defined, and its calculation indicated. The net threshold for instantaneous stimulation is, in general, greater than that for continuous stimulation. The results are discussed with reference to existing theories of reaction times.     

A logical calculus of the ideas immanent in nervous activity

Because of the “all-or-none” character of nervous activity, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiological assumptions are equivalent, in the sense that for every net behaving under one assumption, there exists another net which behaves under the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed.     

An associative memory that can form hypotheses: a phase-coded neural network

 Nonlinear associative memories as realized, e.g., by Hopfield nets are characterized by attractor-type dynamics. When fed with a starting pattern, they converge to exactly one of the stored patterns which is supposed to be most similar. These systems cannot render hypotheses of classification, i.e., render several possible answers to a given classification problem. Inspired by von der Malsburg’s correlation theory of brain function, we extend conventional neural network architectures by introducing additional dynamical variables. Assuming an oscillatory time structure of neural firing, i.e., the existence of neural clocks, we assign a so-called phase to each formal neuron. The phases explicitly describe detailed correlations of neural activities neglected in conventional neural network architectures. Implementing this extension into a simple self-organizing network based on a feature map, we present an associative memory that actually is capable of forming hypotheses of classification. Received: 6 December 1993/Accepted in revised form: 14 July 1994     

A relational theory of the structural changes induced in biological systems by alterations in environment

It is shown that a wide variety of structural alterations in both the “metabolic” and “genetic” apparatus of ( , ℜ)-systems can result from specific changes in the environment of such systems. A number of specific examples are investigated in order to demonstrate the scope of these alterations. Certain biological applications of this discussion are suggested, including a suggestion for a possible interpretation of the mitotic cycle. This research was supported by the United States Air Force through the Air Force Office of Scientific Research of the Air Research and Development Command, under Contract #AF 49 (638)-917.     

Binding and segmentation of multiple objects through neural oscillators inhibited by contour information

 Temporal correlation of neuronal activity has been suggested as a criterion for multiple object recognition. In this work, a two-dimensional network of simplified Wilson-Cowan oscillators is used to manage the binding and segmentation problem of a visual scene according to the connectedness Gestalt criterion. Binding is achieved via original coupling terms that link excitatory units to both excitatory and inhibitory units of adjacent neurons. These local coupling terms are time independent, i.e., they do not require Hebbian learning during the simulations. Segmentation is realized by a two-layer processing of the visual image. The first layer extracts all object contours from the image by means of “retinal cells” with an “on-center” receptive field. Information on contour is used to selectively inhibit Wilson-Cowan oscillators in the second layer, thus realizing a strong separation among neurons in different objects. Accidental synchronism between oscillations in different objects is prevented with the use of a global inhibitor, i.e., a global neuron that computes the overall activity in the Wilson-Cowan network and sends back an inhibitory signal. Simulations performed in a 50×50 neural grid with 21 different visual scenes (containing up to eight objects + background) with random initial conditions demonstrate that the network can correctly segment objects in almost 100% of cases using a single set of parameters, i.e., without the need to adjust parameters from one visual scene to the next. The network is robust with reference to dynamical noise superimposed on oscillatory neurons. Moreover, the network can segment both black objects on white background and vice versa and is able to deal with the problem of “fragmentation.” The main limitation of the network is its sensitivity to static noise superimposed on the objects. Overcoming this problem requires implementation of more robust mechanisms for contour enhancement in the first layer in agreement with mechanisms actually realized in the visual cortex. Received: 25 October 2001 / Accepted: 26 February 2003 / Published online: 20 May 2003 Correspondence to: Mauro Ursino (e-mail: mursino@deis.unibo.it, Tel.: +39-051-2093008, Fax: +39-051-2093073)     

Contribution to the theory of random and biased nets

The probabilistic theory of random and biased nets is further developed by the “tracing” method treated previously. A number of biases expected to be operating in nets, particularly in sociograms, is described. Distribution of closed chain lengths is derived for random nets and for nets with a simple “reflexive” bias. The “island model” bias is treated for the case of two islands and a single axon tracing, resulting in a pair of linear difference equations with two indices. The reflexive bias is extended to multiple-axon tracing by an approximate method resulting in a modification of the random net recursion formula. Results previously obtained are compared with empirical findings and attempts are made to account for observed discrepancies.     

Tuned solutions in dynamic neural fields as building blocks for extended EEG models

The most prominent functional property of cortical neurons in sensory areas are their tuned receptive fields which provide specific responses of the neurons to external stimuli. Tuned neural firing indeed reflects the most basic and best worked out level of cognitive representations. Tuning properties can be dynamic on a short time-scale of fractions of a second. Such dynamic effects have been modeled by localised solutions (also called “bumps” or “peaks”) in dynamic neural fields. In the present work we develop an approximation method to reduce the dynamics of localised activation peaks in systems of n coupled nonlinear d-dimensional neural fields with transmission delays to a small set of delay differential equations for the peak amplitudes and widths only. The method considerably simplifies the analysis of peaked solutions as demonstrated for a two-dimensional example model of neural feature selectivity in the brain. The reduced equations describe the effective interaction between pools of local neurons of several (n) classes that participate in shaping the dynamic receptive field responses. To lowest order they resemble neural mass models as they often form the base of EEG-models. Thereby they provide a link between functional small-scale receptive field models and more coarse-grained EEG-models. More specifically, they connect the dynamics in feature-selective cortical microcircuits to the more abstract local elements used in coarse-grained models. However, beside amplitudes the reduced equations also reflect the sharpness of tuning of the activity in a d-dimensional feature space in response to localised stimuli.     

The pattern of sensory discharge can determine the motor response in young Xenopus tadpoles

Young Xenopus tadpoles were used to test whether the pattern of discharge in specific sensory neurons can determine the motor response of a whole animal. Young Xenopus tadpoles show two main rhythmic behaviours: swimming and struggling. Touch-sensitive skin sensory neurons in the spinal cord of immobilised tadpoles were penetrated singly or in pairs using microelectrodes to allow precise control of their firing patterns. A single impulse in one Rohon-Beard neuron (= light touch) could sometimes trigger “fictive” swimming. Two to six impulses at 30–50 Hz (= a light stroke) reliably triggered fictive swimming. Neither stimulus evoked fictive struggling. Twenty-five or more impulses at 30–50 Hz (= pressure) could evoke a pattern of rhythmic bursts, distinct from swimming and suitable to drive slower, stronger movements. This pattern showed some or all the characteristics of “fictive” struggling. These results demonstrate clearly that sensory neurons can determine the pattern of motor output simply by their pattern of discharge. This provides a simple form of behavioural selection according to stimulus. Accepted: 28 November 1996     

Compact internal representation of dynamic situations: neural network implementing the causality principle

Animals for survival in complex, time-evolving environments can estimate in a “single parallel run” the fitness of different alternatives. Understanding of how the brain makes an effective compact internal representation (CIR) of such dynamic situations is a challenging problem. We propose an artificial neural network capable of creating CIRs of dynamic situations describing the behavior of a mobile agent in an environment with moving obstacles. The network exploits in a mental world model the principle of causality, which enables reduction of the time-dependent structure of real situations to compact static patterns. It is achieved through two concurrent processes. First, a wavefront representing the agent’s virtual present interacts with mobile and immobile obstacles forming static effective obstacles in the network space. The dynamics of the corresponding neurons in the virtual past is frozen. Then the diffusion-like process relaxes the remaining neurons to a stable steady state, i.e., a CIR is given by a single point in the multidimensional phase space. Such CIRs can be unfolded into real space for execution of motor actions, which allows a flexible task-dependent path planning in realistic time-evolving environments. Besides, the proposed network can also work as a part of “autonomous thinking”, i.e., some mental situations can be supplied for evaluation without direct motor execution. Finally we hypothesize the existence of a specific neuronal population responsible for detection of possible time-space coincidences of the animal and moving obstacles.