Methods of analysis of neural nets

The properties of isotropy, smoothness, minimum curvature and locality suggest the shape of filled-in contours between two boundary edges. The contours are composed of the arcs of two circles tangent to the given edges, meeting smoothly, and minimizing the total curvature. It is shown that shapes meeting all the above requirement can be generated by a network which performs simple, local computations. It is suggested that the filling-in process plays an important role in the early processing of visual information.     

A note on neural nets

A neural mechanism is described which provides for a perception which is invariant with respect to movement of the stimulus.     

Self-control in neural nets

This paper deals with the problem of self-controlling nets of Caianiello's type. These nets control themselves through their own elements.     

Contribution to the probabilistic theory of neural nets: IV. Various models for inhibition

Input-output formulas are derived for a neuron upon which converge single axones of two other neurons, which are subjected to a Poisson shower, where a number of different assumptions are made concerning the mechanism of inhibition. In one assumption so-called “bilateral pre-inhibition” is considered. That is to say, both neuronsN ₁ andN ₂ may exciteN ₃, but if the stimulus of one of them follows within a certain interval σ of the other, the second stimulus is not effective. This model is essentially no different from that involving two excitatory neurons acting upon a neuron having a refractory period. Another mechanism considered involves so-called “pre-and-post” inhibition, in which if two stimuli fromN ₁ andN ₂ fall within σ,both are ineffective. This case being mathematically much more involved than the preceding, an approximation method is used for deriving the input-output formula. Previous papers of this series are denoted by I, II, and III in this paper.     

On matrices of neural nets

The structure of a complete or incomplete neural net is represented here by several matrices. The activity equation of the net follows in a general form. A chain or cycle is defined as a neural structure whose connection matrix is unitary. We can compute the number of simple chains by a recurrent formula. Academia Sinica     

Memory topology of neural nets

Aiming at a topology of memory-growth, a computer-simulated mechanism is developed here, based on the probabilistic neural model previously developed by Anninos, Csermeley & Harth.The essential topology of this mechanism reflects the generally accepted idea that memory growth is achieved by means of association processes, i.e. through the cross-referencing of new inputs with already acquired information. In terms of computer-simulation such mechanism is developed by following the formalism of Set Theory. So the memory units, i.e. the neural netlets are equated to subsets of a set which represents a larger memory-system. Consistently with this formalism, association is a necessary, although non-sufficient condition for the growth of memory and the Boolean overlap of two subsets, i.e. of two netlets, is identified with the cross-reference mechanism which we assume to be the basis of memory growth.     

A benchmark for how well neural nets generalize

This paper discusses the problem of extending the domain of learning sets and introduces HERBIE, a program which achieves this through graphical procedures rather than via neural networks. It is argued that for theoretical reasons HERBIE is well-suited to serving as a benchmark for measuring generalization efficacy, and therefore to serving as a means of testing claims of emergent distributed intelligence in neural nets. The successful results of tests of HERBIE as a pattern recognizer are presented, and HERBIE's behavior is favorably compared to neural nets for several real generalization problems. Finally, applications of HERBIE independent of its serving as a generalization benchmark, particularly in the area of cognitive science, are discussed.     

A matrix calculus for neural nets: II

In a previous paper a method was given by which the efferent activity of an idealized neural net could be calculated from a given afferent pattern. Those results are extended in the present paper. Conditions are given under which nets may be considered equivalent. Rules are given for the reduction or extension of a net to an equivalent net. A procedure is given for constructing a net which has the property of converting each of a given set of afferent activity patterns into its corresponding prescribed efferent activity pattern.     

A canonical form for neural nets without circles

By “neural net” will be meant “neural net without circles.” Every neural net effects a transformation from inputs (i.e., firing patterns of the input neurons) to outputs (firing patterns of the output neurons). Two neural nets will be calledequivalent if they effect the same transformation from inputs to outputs. A canonical form is found for neural nets with respect to equivalence; i.e., a class of neural nets is defined, no two of which are equivalent, and which contains a neural net equivalent to any given neural net. This research was supported by the U.S. Air Force under Contract AF 49(638)-414 monitored by the Air Force Office of Scientific Research.     

A diffusion model of a neuron and neural nets

In the paper a diffusion model of a neuron is treated. A new, less restrictive than usually, condition of applicability of a diffusion model is presented. As a result the point-process-to-point-process model of a neuron is obtained, which produces an output signal of the same kind as the accepted input signals.     

Symmetry breaking in neural nets

In this paper two well-known homogeneous models of neural nets undergoing symmetry-breaking transitions are studied in order to see if, after the transition, there is the appearance of Goldstone models. These have been found only in an approximate way; there are indications, however, that they can play a prominent role when the tissue is subjected to external inputs, constraining it to be slaved to the characteristics of those. This circumstance should be essential in explaining how a structured net can store complex inputs and give subsequently ordered outputs.     

The wiring-in of neural nets revisited

Earlier and some recent ideas about the possible modes of specification of the wiring-in of nervous systems are reviewed in the light of older and several recent experiments, and some new ideas are suggested. It is argued that certain general principles, notably the postulated ‘principle of alternative matching’ (PALMA) and a suggested and related ‘kaleidoscopic effect’ (KALEF), as well as the notion of an ‘extracellular guidance network’ (ECGN), are in good agreement with recent and older findings concerning axonal guidance during neural wiring-in. It seems possible that by means of genetically programmed processes, neurons become systematically combinatorially labelled to such a degree that possibly all neurons areuniquely specified, as regards the combination oftypes of cell labels they make. Yet, there remains considerable freedom as regards the modes of arrangements of cell labels within cell surface membranes and the KALEF permits to overcome apparent difficulties that confronted earlier versions of the cell labelling hypotheses (cf. Edelman,Science 219, 450–457, 1983, for mention of such difficulties). Apart from label specification, neural development seems to depend on trophic factors, which are also essential for the maintenance of the developed nervous system. The systematic programmes for cell labelling, apart from generating all the required neurons, also produces inappropriate neurons and synaptic connections. These are got rid of by systematic cell death and/or atrophy of inappropriate synapses and/or elimination of inappropriate axon collaterals. The resulting neural net seems then very specifically wired-in for each species, apparently without redundant neurons.     

Mathematical biophysics of some neural nets

Methods are considered for constructing neural nets of the McCulloch-Pitts type giving arbitrary time delays between stimuli and their responses. By introducing cycles of neurons, the problem is treated from the standpoint of economizing on the number of neurons required. A generalization is effect that will realize any unique temporal response pattern from a unique stimulus.     

Evoked potentials in artificial neural nets

A neural net model of discrete populations of formal neurons have been constructed to study evoked potentials based on our previous simulation studies (Anninos, 1972). Some interesting results came up from the examination of our findings regarding the latencies and the period of the cyclic activity of the evoked potentials. In fact, the different successive latencies for the five identical stimuli and the different periods for each of the cyclic activities, all are consequences of inhibitory and excitatory influences from a large neuronal population. Furthermore, such behavior of the system is not only related to the unknown neuronal population but was also substantially altered by what occurred in other systems at the time of stimulus, or prior to it.Computation assistance was provided by the Health Sciences Computing Facility, UCLA, sponsored by NIH Special Research Resources Grant RR-3. Research was sponsored by NSF Grant GB 30498 and NIH Grant NS-8498.     

Noisy neural nets exhibiting memory domains

Previous studies with probabilistic neural nets in which the neural connections are set up by means of chemical markers, revealed the existence of multiple memory domains. We generalized these studies by considering the intrinsic noise of the systems, caused by the spontaneous release of synaptic transmitter substance. A simple mathematical model is developed, which yields characteristics of multiple memory domains analogous to those occurring in noiseless nets.     

Cyclic modes in artificial neural nets

Artificial neural nets constructed of dicrete populations of 200–1000 formal neurons have been studied through computer simulation. Among the basic assumptions of operation of these nets are the following: a) Each neuron fires at times which are integral multiples of the synaptic delay . b) It produces the appropriate PSP's after . c) All the neurons have the same refractory period and d) temporal summation occurs without decrement, for a period less than the synaptic delay. The nets were specified by a number of parameters: fraction of inhibitory neurons in the population, average number of connections to each cell, threshold for cell firing. These parameters did not determine the detailed microscopical structures of nets which was established separately on a random basis.For the range of the parameters considered in this study it was found that neural nets are capable of supporting self-maintaining activity in the form of cycling modes, characterized by a fixed period. The period of the cycles can be altered by a steady, non-cycling external input to the net. Evidence is presented that the cycling modes depend upon the statistical parameters of the net and the stimulus characteristics rather than on the detailed structure of the net. These results suggest that non-structured nerve nets may respond in specific manner to specific stimuli.

Glossary

Parameters of Neural Net Model Synaptic delay - A Total number of neurons in the netlet - h Fraction of inhibitory neurons in the netlet (in % of total number of neurons) - ⁺ Average number of axon branches emanating from anexcitatory neuron - ^– Average number of axon branches emanating from an inhibitory neuron - k ⁺ Average EPSP produced by an excitatory neuron in arbitrary units of amplitude - k ^– Average IPSP produced by an inhibitory neuron in arbitrary units of amplitude - Firing threshold of neurons in the netlet - The minimum number of ESPS's necessary to trigger a neuron in the absence of inhibitory inputs - The minimum number of ESPS's necessary to trigger a neuron in the presence of inhibitory inputs. Dynamic Parameters of the Model n An integer giving the number of elapsed synaptic delays (i.e. elapsed time) - _n The activity; i.e. the fraction of active neurons in the netlet at t=n (the actual number of active cells is given by _nA) - _n={ⁱ_n} State vector of single netlet at time n This research has been supported by NIH grants NS-8012 and NS-8498, and NSF grant GB-30498. Computation assistance was provoded by the Health Sciences Computing Facility, UCLA, sponsored by NIH Special Research Resources grant RR-3.     

A new model for simple neural nets and its application in the design of a neural oscillator

The results of a previous theoretical study of a class of systems are applied for the design of neural nets which try to simulate biological behavior. Besides the models for single aperiodic and periodic neurons, a “neural oscillator” is developed which consists of two cross-excited neurons. Its response is similar to the firing pattern of certain biological neural oscillators, like the flying system of the locust. Also, by proper change of its parameters, it can be made highly irregular, providing a deterministic model for the spontaneous neural activity.     

CDMA and TDMA based neural nets

CDMA and TDMA telecommunication techniques were established long time ago, but they have acquired a renewed presence due to the rapidly increasing mobile phones demand. In this paper, we are going to see they are suitable for neural nets, if we leave the concept "connection" between processing units and we adopt the concept "messages" exchanged between them. This may open the door to neural nets with a higher number of processing units and flexible configuration.