A canonical form for neural nets without circles

By “neural net” will be meant “neural net without circles.” Every neural net effects a transformation from inputs (i.e., firing patterns of the input neurons) to outputs (firing patterns of the output neurons). Two neural nets will be calledequivalent if they effect the same transformation from inputs to outputs. A canonical form is found for neural nets with respect to equivalence; i.e., a class of neural nets is defined, no two of which are equivalent, and which contains a neural net equivalent to any given neural net. This research was supported by the U.S. Air Force under Contract AF 49(638)-414 monitored by the Air Force Office of Scientific Research.     

Steady states in random nets: I

A neural net is taken to consist of a semi-infinite chain of neurons with connections distributed according to a certain probability frequency of the lengths of the axones. If an input of excitation is “fed” into the net from an outside source, the statistical properties of the net determine a certain steady state output. The general functional relation between the input and the output is derived as an integral equation. For a certain type of probability distribution of connections, this equation is reducible to a differential equation. The latter can be solved by elementary methods for the output in terms of the input in general and for the input in terms of the output in special cases.     

A statistical approach to the theory of the central nervous system

A “probabilistic” rather than a “deterministic” approach to the theory of neural nets is developed. Neural nets are characterized by certain parameters which give the probability distributions of different kinds of synaptic connections throughout the net. Given a “state” of the net (i.e., the distribution of firing neurons) at a given moment, an equation for the state at the next moment of quantized time is deduced. Certain very special cases involving constant distributions are solved. A necessary condition for a steady state is deduced in terms of an integral equation, in general non-linear.     

Response time and threshold of a random net

The response time of a random net is defined as the expected time (measured in the number of synaptic delays) required for the excitation in the net (measured by the fraction of neurons firing per unit time) to reach a certain level. The response time is calculated in terms of the net parameters as a function of the intensity of the outside stimulation. Two principal types of cases are studied, 1) an instantaneous initial stimulation, and 2) continuously applied stimulation. It is shown that for a certain type of net where the required level of excitation is small, the response time-intensity equation reduces to the one derived on the basis of the “one-factor” theory applied to a neural connection. More general assumptions, however, give different types of equations. The concept of the “net threshold” is defined, and its calculation indicated. The net threshold for instantaneous stimulation is, in general, greater than that for continuous stimulation. The results are discussed with reference to existing theories of reaction times.     

The neural crest and neural crest cells: discovery and significance for theories of embryonic organization

The neural crest has long fascinated developmental biologists, and, increasingly over the past decades, evolutionary and evolutionary developmental biologists. The neural crest is the name given to the fold of ectoderm at the junction between neural and epidermal ectoderm in neurula-stage vertebrate embryos. In this sense, the neural crest is a morphological term akin to head fold or limb bud. This region of the dorsal neural tube consists of neural crest cells, a special population(s) of cell, that give rise to an astonishing number of cell types and to an equally astonishing number of tissues and organs. Neural crest cell contributions may be direct — providing cells — or indirect — providing a necessary, often inductive, environment in which other cells develop. The enormous range of cell types produced provides an important source of evidence of the neural crest as a germ layer, bringing the number of germ layers to four — ectoderm, endoderm, mesoderm, and neural crest. In this paper I provide a brief overview of the major phases of investigation into the neural crest and the major players involved, discuss how the origin of the neural crest relates to the origin of the nervous system in vertebrate embryos, discuss the impact on the germ-layer theory of the discovery of the neural crest and of secondary neurulation, and present evidence of the neural crest as the fourth germ layer. A companion paper (Hall, Evol. Biol. 2008) deals with the evolutionary origins of the neural crest and neural crest cells.     

A matrix calculus for neural nets: II

In a previous paper a method was given by which the efferent activity of an idealized neural net could be calculated from a given afferent pattern. Those results are extended in the present paper. Conditions are given under which nets may be considered equivalent. Rules are given for the reduction or extension of a net to an equivalent net. A procedure is given for constructing a net which has the property of converting each of a given set of afferent activity patterns into its corresponding prescribed efferent activity pattern.     

Context-dependent associations in linear distributed memories

In this article we present a method that allows conditioning of the response of a linear distributed memory to a variable context. This method requires a system of two neural networks. The first net constructs the Kronecker product between the vector input and the vector context, and the second net supports a linear associative memory. This system is easily adaptable for different goals. We analyse here its capacity for the conditional extraction of features from a complex perceptual input, its capacity to perform quasi-logical operations (for instance, of the kind of “exclusive-or”), and its capacity to structurate a memory for temporal sequences which access is conditioned by the context. Finally, we evaluate the potential importance of the capacity to establish arbitrary contexts, for the evolution of biological cognitive systems. Part of this study has been presented in a preliminary version at the XVI Reunión Científica de la Sociedad Argentina de Biofísica, Tigre, Argentina, December 1987.     

The Physical Chemistry of Brain and Neural Cell Membranes: An Overview

The formation of cell membranes through the physical–chemical interaction of two hydrophilic colloidal fluids is applied to the formation of the membranes of brain and neural cells. Also described is the membrane mechanism of transfer of ions and compounds necessary for brain and neural cell functions into the cerebrospinal fluid through the blood–brain barrier. Changes in the cerebrospinal fluid giving rise to degradation of brain and neural cells and the formation of precipitates within the brain are considered. Monitoring of electrolyte changes in metabolic fluids is shown to be a possible method of predicting the onset of degenerate brain conditions.     

Application of information networks to a theory of vision

Some principles of information theory are utilized in the design of neural nets of the McCulloch-Pitts type. In particular, problems are considered where signals from several neurons must pass through a single one, thus resulting in a “bottleneck” in the flow of information, an abstract model of the corresponding bottleneck from the retina to the optic nerve. The first part of the paper deals with a construction of a McCulloch-Pitts net in which the redundancy in the messages originating in two neurons is utilized so that the messages can be sent over a single neuron with little loss of information. In the second part, messages from a set of neurons are “pumped” into two channel neurons. The optimum connection scheme is computed for this case, i.e, one resulting in a minimum loss of information. Possible biological implications of this approach are indicated.     

The representation of biological systems from the standpoint of the theory of categories

A mathematical framework for a rigorous theory of general systems is constructed, using the notions of the theory of Categories and Functors introduced by Eilenberg and MacLane (1945,Trans. Am. Math. Soc.,58, 231–94). A short discussion of the basic ideas is given, and their possible application to the theory of biological systems is discussed. On the basis of these considerations, a number of results are proved, including the possibility of selecting a unique representative (a “canonical form”) from a family of mathematical objects, all of which represent the same system. As an example, the representation of the neural net and the finite automaton is constructed in terms of our general theory.     

Measuring Nitrogen and Phosphorus Uptake by Intact Roots of Mature Acer saccharum Marsh., Pinus resinosa Ait., and Picea abies (L.) Karst

A common method for measuring uptake by intact roots in situ is the depletion method, wherein intact fine roots are separated from soil and placed in nutrient solution. The difference between initial and final amounts of nutrient in solution is attributed to root uptake. Variations on this method include applying pretreatment solutions, training roots to grow into bags or trays, and varying concentrations of nutrient solution. We tested whether variations in methods affected measured net uptake rates of NH ₄ ⁺ , NO ₃ ⁻ , and PO ₄ ³⁻ . Intact roots of 60 year-old sugar maple (Acer saccharum Marsh.), red pine (Pinus resinosa Ait.), and Norway spruce (Picea abies (L.) Karst.) were given one of four treatments prior to measuring net uptake. “Trained” roots were grown in a sand-soil mixture. “Recovered” roots were excavated and allowed to recover in nutrient solution for two or four days (“two-day recovery” and “four-day recovery”, respectively). “No recovery” roots were excavated and used immediately in experiments. We exposed roots to three concentrations of nutrient solutions to observe the effects of initial nutrient solution concentration. Initial nutrient solution concentration was an important source of variation in measured uptake rates, and N uptake was stimulated by low antecedent concentrations. We found no significant differences in net uptake rates between pretreatments for any of the species studied, indicating that our pretreatments were not effective in improving measurement of uptake. Such pretreatments may not be necessary for measuring net uptake and may not hinder the comparison of rates measured using variations of the depletion method.     

Effect of Temperature on Net Assimilation Rate

Net assimilation rates and other growth attributes were comparedfor rape, sunflower, and maize plants growing widely spacedat temperatures of 10°, 16°, 22°, 28°, and 34°C, in light of 3, 000 f.c. intensity. The optimum temperature for net assimilation rate lay between20° and 30° C, and was lowest for rape and highest formaize. The temperature coefficient of the net assimilation ratewas lower than that of the relative growth-rate, especiallyin rape and sunflower, corresponding to an increase in leaf-arearatio with in temperature. This arose to an increase in leaf-arearatio with rise in temperature. This mcrease arose through changeinleafarea/leaf weight; temperature had little effect on leafweight/plant weight. In moderate to warm conditions the net assimilation rate variedlittle with temperature: by only± 10 per cent between12° and 30° C for rape, and 23° and 36° C formaize. This agrees with observations in natural climates whichsuggest that temperature is generally less important than lightin controlling net assimilation rates, except in cool climates.In natural climates, as in these controlled climates, relativegrowth-rate is more temperature-dependent.     

A method of analysis of the models of neural nets

Taking into account Caianiello's work of 1961 a model of a neuron quite similar to his is proposed and studied. For this model, where a temporal summation and a period of refractoriness are assumed, a mathematical approach and a simulation on computer were realized. Particular types of nets were used, namely: nets with topological structures, and fully random nets. The difference between the two types is that the first type has a two-dimensional square structure and depends on the rules of the formation of connection between the neurons, while the second type is realized by means of the probability distribution function governing the formation of the structure of the net.These types of neural nets are analysed by means of a method which permits to obtain various parameters which characterize their behaviour in time and space in terms of the trajectory of the system. Many experiments are also reported; the statistical analyses, made on them, show the great importance and influence of refractoriness on the behaviour of neural networks.In the last part of the work an interesting case is reported, in which the reaction of the net to a disturbance shows that a kind of adaptation takes place, although the structure of the net stays unchanged.On leave of absence from the Lithuanian Academy of Sciences, Vilnius, Lithuanian S.S.R.     

Outline of a probabilistic approach to animal sociology: I

The non-transitive character of he peck right relation gives rise to different “social structures” in an aggregate of individuals. A method is developed for computing the respective probabilities of occurrence for each type of structure in small aggregates on the basis of random determination of peck right between each pair of individuals.     

Neural networks in chemosystematic studies of Asteraceae: a classification based on a dichotomic approach

This paper describes the application of artificial neural nets as an alternative and efficient method for the classification of botanical taxa based on chemical data (chemosystematics). A total of 28,000 botanical occurrences of chemical compounds isolated from the Asteraceae family were chosen from the literature, and grouped by chemical class for each species. Four tests were carried out to differentiate and classify different botanical taxa. The qualifying capacity of the artificial neural nets was dichotomically tested at different hierarchical levels of the family, such as subfamilies and groups of Heliantheae subtribes. Furthermore, two specific subtribes of the Heliantheae and two genera of one of these subtribes were also tested. In general, the artificial neural net gave rise to good results, with multiple-correlation values R>0.90. Hence, it was possible to differentiate the dichotomic character of the botanical taxa studied.     

A logical calculus of the ideas immanent in nervous activity

Because of the “all-or-none” character of nervous activity, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiological assumptions are equivalent, in the sense that for every net behaving under one assumption, there exists another net which behaves under the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed.     

A logical calculus of the ideas immanent in nervous activity

Because of the “all-or-none” character of nervous activity, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiological assumptions are equivalent, in the sense that for every net behaving under one assumption, there exists another net which behaves under the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed. Reprinted from theBulletin of Mathematical Biophysics, Vol. 5, pp. 115–133 (1943).     

Time series predictions with neural nets: Application to airborne pollen forecasting

Pollen allergy is a common disease causing rhinoconjunctivitis (hay fever) in 5–10% of the population. Medical studies have indicated that pollen related diseases could be highly reduced if future pollen contents in the air could be predicted. In this work we have developed a new forecasting method that applies the ability of neural nets to predict the future behaviour of chaotic systems in order to make accurate predictions of the airborne pollen concentration. The method requires that the neural net be fed with non-zero values, which restricts the method predictions to the period following the start of pollen flight. The operational method outlined here constitutes a different point of view with respect to the more generally used forecasts of time series analysis, which require input of many meteorological parameters. Excellent forecasts were obtained training a neural net by using only the time series pollen concentration values.     

Applications of Perron-Frobenius theory to population dynamics

 By the use of Perron–Frobenius theory, simple proofs are given of the Fundamental Theorem of Demography and of a theorem of Cushing and Yicang on the net reproductive rate occurring in matrix models of population dynamics. The latter result, which is closely related to the Stein–Rosenberg theorem in numerical linear algebra, is further refined with some additional nonnegative matrix theory. When the fertility matrix is scaled by the net reproductive rate, the growth rate of the model is $1$. More generally, we show how to achieve a given growth rate for the model by scaling the fertility matrix. Demographic interpretations of the results are given. Received: 15 January 2000 / Revised version: 15 April 2001 / Published online: 8 May 2002     

Delayed-exponential approximation of a linear homogeneous diffusion model of neuron

The diffusion models of neuronal activity are general yet conceptually simple and flexible enough to be useful in a variety of modeling problems. Unfortunately, even simple diffusion models lead to tedious numerical calculations. Consequently, the existing neural net models use characteristics of a single neuron taken from the pre-diffusion era of neural modeling. Simplistic elements of neural nets forbid to incorporate a single learning neuron structure into the net model. The above drawback cannot be overcome without the use of the adequate structure of the single neuron as an element of a net. A linear (not necessarily homogeneous) diffusion model of a single neuron is a good candidate for such a structure, it must, however, be simplified. In the paper the structure of the diffusion model of neuron is discussed and a linear homogeneous model with reflection is analyzed. For this model an approximation is presented, which is based on the approximation of the first passage time distribution of the Ornstein-Uhlenbeck process by the delayed (shifted) exponential distribution. The resulting model has a simple structure and has a prospective application in neural modeling and in analysis of neural nets.Work supported by Polish Academy of Sciences grant # CPBP 04.01