Outline of a matrix calculus for neural nets

The activity of a neural net is represented in terms of a matrix vector equation with a normalizing operator in which the matrix represents only the complete structure of the net, and the normalized vector-matrix product represents the activity of all the non-afferent neurons. The activity vectors are functions of a quantized time variable whose elements are zero (no activity) or one (activity). Certain properties of the structure matrix are discussed and the computational procedure which results from the matrix vector equation is illustrated by a specific example.     

Delayed-exponential approximation of a linear homogeneous diffusion model of neuron

The diffusion models of neuronal activity are general yet conceptually simple and flexible enough to be useful in a variety of modeling problems. Unfortunately, even simple diffusion models lead to tedious numerical calculations. Consequently, the existing neural net models use characteristics of a single neuron taken from the pre-diffusion era of neural modeling. Simplistic elements of neural nets forbid to incorporate a single learning neuron structure into the net model. The above drawback cannot be overcome without the use of the adequate structure of the single neuron as an element of a net. A linear (not necessarily homogeneous) diffusion model of a single neuron is a good candidate for such a structure, it must, however, be simplified. In the paper the structure of the diffusion model of neuron is discussed and a linear homogeneous model with reflection is analyzed. For this model an approximation is presented, which is based on the approximation of the first passage time distribution of the Ornstein-Uhlenbeck process by the delayed (shifted) exponential distribution. The resulting model has a simple structure and has a prospective application in neural modeling and in analysis of neural nets.Work supported by Polish Academy of Sciences grant # CPBP 04.01     

Serotonin-immunoreactive neural system and contractile system in the hydroid Cladonema (Cnidaria, Hydrozoa)

Serotonin is a widespread neurotransmitter which is present in almost all animal phyla including lower metazoans such as Cnidaria. Serotonin detected in the polyps of several cnidarian species participates in the functioning of a neural system. It was suggested that serotonin coordinates polyp behavior. For example, serotonin may be involved in muscle contraction and/or cnidocyte discharge. However, the role of serotonin in cnidarians is not revealed completely yet. The aim of this study was to investigate the neural system of Cladonema radiatum polyps. We detected the net of serotonin-positive processes within the whole hydranth body using anti-serotonin antibodies. The hypostome and tentacles had denser neural net in comparison with the gastric region. Electron microscopy revealed muscle processes throughout the hydranth body. Neural processes with specific vesicles and neurotubules in their cytoplasm were also shown at an ultrastructural level. This work demonstrates the structure of serotonin-positive neural system and smooth muscle layer in C. radiatum hydranths.     

The construction of a simultaneous functional order in nervous systems

The functional order of a collection of nervous elements is available to the system itself, as opposed to the anatomical geometrical order which exists only for external observers. It has been shown before (Part I) that covariances or coincidences in the signal activity of a neural net can be used in the construction of a simultaneous functional order in which a modality is represented as a concatenation of districts with a lattice structure. In this paper we will show how the resulting functional order in a nervous net can be related to the geometry of the underlying detector array. In particular, we will present an algorithm to construct an abstract geometrical complex from this functional order. The algebraic structure of this complex reflects the topological and geometrical structure of the underlying detector array. We will show how the activated subcomplexes of a complex can be related to segments of the detector array that are activated by the projection of a stimulus pattern. The homology of an abstract complex (and therefore of all of its subcomplexes) can be obtained from simple combinatorial operations on its coincidence scheme. Thus, both the geometry of a detector array and the topology of projections of stimulus patterns may have an objective existence for the neural system itself.     

A canonical form for neural nets without circles

By “neural net” will be meant “neural net without circles.” Every neural net effects a transformation from inputs (i.e., firing patterns of the input neurons) to outputs (firing patterns of the output neurons). Two neural nets will be calledequivalent if they effect the same transformation from inputs to outputs. A canonical form is found for neural nets with respect to equivalence; i.e., a class of neural nets is defined, no two of which are equivalent, and which contains a neural net equivalent to any given neural net. This research was supported by the U.S. Air Force under Contract AF 49(638)-414 monitored by the Air Force Office of Scientific Research.     

A method of analysis of the models of neural nets

Taking into account Caianiello's work of 1961 a model of a neuron quite similar to his is proposed and studied. For this model, where a temporal summation and a period of refractoriness are assumed, a mathematical approach and a simulation on computer were realized. Particular types of nets were used, namely: nets with topological structures, and fully random nets. The difference between the two types is that the first type has a two-dimensional square structure and depends on the rules of the formation of connection between the neurons, while the second type is realized by means of the probability distribution function governing the formation of the structure of the net.These types of neural nets are analysed by means of a method which permits to obtain various parameters which characterize their behaviour in time and space in terms of the trajectory of the system. Many experiments are also reported; the statistical analyses, made on them, show the great importance and influence of refractoriness on the behaviour of neural networks.In the last part of the work an interesting case is reported, in which the reaction of the net to a disturbance shows that a kind of adaptation takes place, although the structure of the net stays unchanged.On leave of absence from the Lithuanian Academy of Sciences, Vilnius, Lithuanian S.S.R.     

A matrix calculus for neural nets: II

In a previous paper a method was given by which the efferent activity of an idealized neural net could be calculated from a given afferent pattern. Those results are extended in the present paper. Conditions are given under which nets may be considered equivalent. Rules are given for the reduction or extension of a net to an equivalent net. A procedure is given for constructing a net which has the property of converting each of a given set of afferent activity patterns into its corresponding prescribed efferent activity pattern.     

A computer simulation of a neuron net model as a self-organizing system

A neural net model is simulated on an IBM-1130 digital computer. The model includes rules for learning of the presented patterns. The learning algorithm uses an iteration procedure, in order to compute the ultimate cross coupling-coefficients between the neurons for a specific pattern. The network has a set of latent cyclic modes or reverberations. If the net is stimulated briefly, by presenting a pattern, it will subsequently either return to quiescence or settle into periodic activity in one of its cyclic modes.     

The construction of a simultaneous functional order in nervous systems

The signal activity in a neural net will be constrained both by its physical structure and by environmental constraints. By monitoring its signal activity a neural system can build up a simultaneous functional order that encodes these constraints. We have previously (Part I) presented two models that construct a simultaneous functional order in a collection of neural elements using either signal-covariances or signal-coincides. In this paper we present the results of simulation experiments that were performed to study the influence of the physical constraints of a neural system on the simultaneous functional order produced by both models. In the simulation experiments we used a one-dimensional detector array. We delineate the physical constraints such an array has to satisfy in order to induce a functional order relation that allows an isomorphism with a geometrical order. We show that for an appropriate choice of the system parameters both models can produce a simultaneous functional order with sufficient internal coherence to allow isomorphisms with a triangulation. In this case the dimensionality and the coherence of the detector array are objectively available to the system itself.     

Implementation of a pulse coupled neural network in FPGA

The Pulse Coupled neural network, PCNN, is a biologically inspired neural net and it can be used in various image analysis applications, e.g. time-critical applications in the field of image pre-processing like segmentation, filtering, etc. a VHDL implementation of the PCNN targeting FPGA was undertaken and the results presented here. The implementation contains many interesting features. By pipelining the PCNN structure a very high throughput of 55 million neuron iterations per second could be achieved. By making the coefficients re-configurable during operation, a complete recognition system could be implemented on one, or maybe two, chip(s). Reconsidering the ranges and resolutions of the constants may save a lot of hardware, since the higher resolution requires larger multipliers, adders, memories etc.     

Artificial neural networks and their use in quantitative pathology

A brief general introduction to artificial neural networks is presented, examining in detail the structure and operation of a prototype net developed for the solution of a simple pattern recognition problem in quantitative pathology. The process by which a neural network learns through example and gradually embodies its knowledge as a distributed representation is discussed, using this example. The application of neurocomputer technology to problems in quantitative pathology is explored, using real-world and illustrative examples. Included are examples of the use of artificial neural networks for pattern recognition, database analysis and machine vision. In the context of these examples, characteristics of neural nets, such as their ability to tolerate ambiguous, noisy and spurious data and spontaneously generalize from known examples to handle unfamiliar cases, are examined. Finally, the strengths and deficiencies of a connectionist approach are compared to those of traditional symbolic expert system methodology. It is concluded that artificial neural networks, used in conjunction with other nonalgorithmic artificial intelligence techniques and traditional algorithmic processing, may provide useful software engineering tools for the development of systems in quantitative pathology.     

Neural network modeling in optimisation of continuous fermentation processes

The capability of self-recurrent neural networks in dynamic modeling of continuous fermentation is investigated in this simulation study. In the past, feedforward neural networks have been successfully used as one-step-ahead predictors. However, in steady-state optimisation of continuous fermentations the neural network model has to be iterated to predict many time steps ahead into the future in order to get steady-state values of the variables involved in objective cost function, and this iteration may result in increasing errors. Therefore, as an alternative to classical feedforward neural network trained by using backpropagation method, self-recurrent multilayer neural net trained by backpropagation through time method was chosen in order to improve accuracy of long-term predictions. Prediction capabilities of the resulting neural network model is tested by implementing this into the Integrated System Optimisation and Parameter Estimation (ISOPE) optimisation algorithm. Maximisation of cellular productivity of the baker's yeast continuous fermentation was used as the goal of the proposed optimising control problem. The training and prediction results of proposed neural network and performances of resulting optimisation structure are demonstrated.     

Prediction of occurrences of diverse chemical classes in the Asteraceae through artificial neural networks

The training and the application of a neural network system for the prediction of occurrences of secondary metabolites belonging to diverse chemical classes in the Asteraceae is described. From a database containing about 604 genera and 28,000 occurrences of secondary metabolites in the plant family, information was collected encompassing nine chemical classes and their respective occurrences for training of a multi-layer net using the back-propagation algorithm. The net supplied as output the presence or absence of the chemical classes as well as the number of compounds isolated from each taxon. The results provided by the net from the presence or absence of a chemical class showed a 89% hit rate; by excluding triterpenes from the analysis, only 5% of the genera studied exhibited errors greater than 10%.     

The Surface Charge of Mesodermal and Neural Cells of Triturus Embryos

The electrical surface charge of disaggregated mesodermal and neural cells of the neurula stage of Triturus vulgaris embryos is estimated by using the apparatus described in Figure 1. The results showed a significant difference between the net charges of these two cell types: the neural cells had a clearly negative charge, whereas the mesodermal cells seemed to be more or less neutral.
The difference of the surface charges may partly explain the "sorting-out" phenomenon in the mixed reaggregates of the disaggregated cells.     

The group structure of some neural nets

A method is given for using a set of disjoint cycles as the main body of a neural net that will given rise to more than one temporal response pattern for different afferent stimuli. The method arises out of considering the correspondence between this type of net and certain Abelian groups of finite order. The consideration also gives rise to a possible definition of the “complexity” of this type of neural net.     

Use of a neural net computer system for analysis of flow cytometric data of phytoplankton populations

Flow cytometry has been used over the past 5 years to begin detailed exploration of the distribution and abundance of picoplankton in the oceans. Light scattering and fluorescence measurements on individual plankton cells in seawater samples allow construction of population signatures from size and pigment characteristics. The use of "list mode" data has made these studies possible, but on-shore analysis of copious data does not permit on-site reexamination of important or unexpected observations, and overall effort is greatly handicapped by data analysis time. Here we describe the application of neural net computer technology to the analysis of flow cytometry data. Although the data used in this study are from oceanographic research, the results are general and should be directly applicable to flow cytometry data of any sort. Neural net computers are ideally suited to perform the pattern recognition required for the quantitative analysis of flow cytometry data. Rather than being programmed to perform analysis, the neural net computer is "taught" how to analyze the cell populations by presenting examples of inputs and correct results. Once the system is "trained," similar data sets can be analyzed rapidly and objectively, minimizing the need for laborious user interaction. The neural network described here offers the advantages of 1) adaptability to changing conditions and 2) potential real-time analysis. High accuracy and processing speed near that required for real-time classification have been achieved in a software simulation of the neural network on a Macintosh SE personal computer.     

The differentiation characteristics of neural derivatives in rat teratomas

Peculiarities of differentiation in teratomas obtained by transplantation of 9- and 11-day-old Wistar rat embryos under testicle capsule of adult rats. The main portion of the teratoma is represented by neural derivatives, i.e. by neural tissue foci with pyramidal, stellate and fusiform neural cells, glial cells, and a developed capillary net. Nervous fibers and ganglia-like structures have also been revealed. At the same time, cytoarchitectonics of cerebral cortex does not form in neural areas.     

Efficient protein crystallization

High-throughput molecular biology and crystallography advances have placed an increasing demand on crystallization, the one remaining bottleneck in macromolecular crystallography. This paper describes three experimental approaches, an incomplete factorial crystallization screen, a high-throughput nanoliter crystallization system, and the use of a neural net to predict crystallization conditions via a small sample (approximately 0.1%) of screening results. The use of these technologies has the potential to reduce time and sample requirements. Initial experimental results indicate that the incomplete factorial design detects initial crystallization conditions not previously discovered using commercial screens. This may be due to the ability of the incomplete factorial screen to sample a broader portion of "crystallization space," using a multidimensional set of components, concentrations, and physical conditions. The incomplete factorial screen is complemented by a neural network program used to model crystallization. This capability is used to help predict new crystallization conditions. An automated, nanoliter crystallization system, with a throughput of up to 400 conditions/h in 40-nl droplets (total volume), accommodates microbatch or traditional "sitting-drop" vapor diffusion experiments. The goal of this research is to develop a fully-automated high-throughput crystallization system that integrates incomplete factorial screen and neural net capabilities.     

Stability and parameter dependency analysis of a facilitation tectal column (FTC) model

Mathematical models and computer simulations have been widely used to study the spatio-temporal characteristics of the processing of information carried out by the central nervous system. When trying to show whether or not a neural model accounts for the phenomena under study, if the number of parameters whose values need to be calculated becomes large, then computer simulations alone become very inefficient to define such values. Here, we developed stability and parameter dependency analyses of the mathematical representation of a single facilitation tectal column (FTC) model, to show how by using techniques from non-linear systems theory we can define the ranges of parameter values under which the model would explain the required performance of the neural net model. The benefits of these analyses can be grouped in two parts: first, the advantage of using non-linear systems techniques to analyze, analytically, the dynamics of neural net models; and second, we get a deeper understanding of why the hypotheses embedded in the models yield the appropriate behaviors and what are the critical situations (parametric combinations) under which these behaviors are displayed.