Mathematical biophysics of some neural nets

Methods are considered for constructing neural nets of the McCulloch-Pitts type giving arbitrary time delays between stimuli and their responses. By introducing cycles of neurons, the problem is treated from the standpoint of economizing on the number of neurons required. A generalization is effect that will realize any unique temporal response pattern from a unique stimulus.     

Tuning diversity in bagged ensembles

In this paper, we investigate how the level of diversity amongst individual neural networks in a bagged ensemble can significantly influence overall ensemble generalization performance. We propose a new technique that tunes this diversity so that ensemble generalization performance is optimized and evaluate its performance on benchmark regression data-sets.     

A canonical form for neural nets without circles

By “neural net” will be meant “neural net without circles.” Every neural net effects a transformation from inputs (i.e., firing patterns of the input neurons) to outputs (firing patterns of the output neurons). Two neural nets will be calledequivalent if they effect the same transformation from inputs to outputs. A canonical form is found for neural nets with respect to equivalence; i.e., a class of neural nets is defined, no two of which are equivalent, and which contains a neural net equivalent to any given neural net. This research was supported by the U.S. Air Force under Contract AF 49(638)-414 monitored by the Air Force Office of Scientific Research.     

A neural net model for multiple memory domains

Previous studies with neural nets constructed of discrete populations of formal neurons have assumed that all neurons have the same probability of connection with any other neuron in the net. However, in this new study we incorporate the behavior of the neural systems in which the neural connections can be set up by means of chemical markers carried by the individual cells. With this new approach we studied the dynamics of isolated neural nets again as well as the dynamics of neural nets with sustained inputs. Results obtained with this approach show simple and multiple hysteresis phenomena. Such hysteresis loops may be considered to represent the basis for short-term memory.     

The dynamics of symmetric nets

The role of symmetry in simplifying the theory of complex neural systems is argued. When the structural symmetries of a network are expressed as an ismorphism group, implications emerge for the dynamics. Various qualitative possibilities concerning stability of uniform motion in homogeneous nets are discussed and an approach to neural hierarchies is outlined.     

On problem solving with Hopfield neural networks

Hopfield and Tank have shown that neural networks can be used to solve certain computationally hard problems, in particular they studied the Traveling Salesman Problem (TSP). Based on network simulation results they conclude that analog VLSI neural nets can be promising in solving these problems. Recently, Wilson and Pawley presented the results of their simulations which contradict the original results and cast doubts on the usefulness of neural nets. In this paper we give the results of our simulations that clarify some of the discrepancies. We also investigate the scaling of TSP solutions found by neural nets as the size of the problem increases. Further, we consider the neural net solution of the Clustering Problem, also a computationally hard problem, and discuss the types of problems that appear to be well suited for a neural net approach.     

Colon cancer prediction with genetic profiles using intelligent techniques

Micro array data provides information of expression levels of thousands of genes in a cell in a single experiment. Numerous efforts have been made to use gene expression profiles to improve precision of tumor classification. In our present study we have used the benchmark colon cancer data set for analysis. Feature selection is done using t‐statistic. Comparative study of class prediction accuracy of 3 different classifiers viz., support vector machine (SVM), neural nets and logistic regression was performed using the top 10 genes ranked by the t‐statistic. SVM turned out to be the best classifier for this dataset based on area under the receiver operating characteristic curve (AUC) and total accuracy. Logistic Regression ranks as the next best classifier followed by Multi Layer Perceptron (MLP). The top 10 genes selected by us for classification are all well documented for their variable expression in colon cancer. We conclude that SVM together with t-statistic based feature selection is an efficient and viable alternative to popular techniques.     

Recognition of distantly related protein sequences using conserved motifs and neural networks.

A sensitive technique for protein sequence motif recognition based on neural networks has been developed. It involves three major steps. (1) At each appropriate alignment position of a set of N matched sequences, a set of N aligned oligopeptides is specified with preselected window length. N neural nets are subsequently and successively trained on N-1 amino acid spans after eliminating each ith oligopeptide. A test for recognition of each of the ith spans is performed. The average neural net recognition over N such trials is used as a measure of conservation for the particular windowed region of the multiple alignment. This process is repeated for all possible spans of given length in the multiple alignment. (2) The M most conserved regions are regarded as motifs and the oligopeptides within each are used to train intensively M individual neural networks. (3) The M networks are then applied in a search for related primary structures in a databank of known protein sequences. The oligopeptide spans in the database sequence with strongest neural net output for each of the M networks are saved and then scored according to the output signals and the proper combination that follows the expected N- to C-terminal sequence order. The motifs from the database with highest similarity scores can then be used to retrain the M neural nets, which can be subsequently utilized for further searches in the databank, thus providing even greater sensitivity to recognize distant familial proteins. This technique was successfully applied to the integrase, DNA-polymerase and immunoglobulin families.     

Ensembles of neural networks based on the alteration of input feature values

An ensemble performs well when the component classifiers are diverse yet accurate, so that the failure of one is compensated for by others. A number of methods have been investigated for constructing ensemble in which some of them train classifiers with the generated patterns. This study investigates a new technique of training pattern generation. The method alters input feature values of some patterns using the values of other patterns to generate different patterns for different classifiers. The effectiveness of neural network ensemble based on the proposed technique was evaluated using a suite of 25 benchmark classification problems, and was found to achieve performance better than or competitive with related conventional methods. Experimental investigation of different input values alteration techniques finds that alteration with pattern values in the same class is better for generalization, although other alteration techniques may offer more diversity.     

Probabilistic computation by neuromine networks

In this paper, we address the question, can biologically feasible neural nets compute more than can be computed by deterministic polynomial time algorithms? Since we want to maintain a claim of plausibility and reasonableness we restrict ourselves to algorithmically easy to construct nets and we rule out infinite precision in parameters and in any analog parts of the computation. Our approach is to consider the recent advances in randomized algorithms and see if such randomized computations can be described by neural nets. We start with a pair of neurons and show that by connecting them with reciprocal inhibition and some tonic input, then the steady-state will be one neuron ON and one neuron OFF, but which neuron will be ON and which neuron will be OFF will be chosen at random (perhaps, it would be better to say that microscopic noise in the analog computation will be turned into a megascale random bit). We then show that we can build a small network that uses this random bit process to generate repeatedly random bits. This random bit generator can then be connected with a neural net representing the deterministic part of randomized algorithm. We, therefore, demonstrate that these neural nets can carry out probabilistic computation and thus be less limited than classical neural nets.     

CDMA and TDMA based neural nets

CDMA and TDMA telecommunication techniques were established long time ago, but they have acquired a renewed presence due to the rapidly increasing mobile phones demand. In this paper, we are going to see they are suitable for neural nets, if we leave the concept "connection" between processing units and we adopt the concept "messages" exchanged between them. This may open the door to neural nets with a higher number of processing units and flexible configuration.     

Noisy neural nets exhibiting memory domains

Previous studies with probabilistic neural nets in which the neural connections are set up by means of chemical markers, revealed the existence of multiple memory domains. We generalized these studies by considering the intrinsic noise of the systems, caused by the spontaneous release of synaptic transmitter substance. A simple mathematical model is developed, which yields characteristics of multiple memory domains analogous to those occurring in noiseless nets.     

On the ability of neural networks to perform generalization by induction

The ability of neural networks to perform generalization by induction is the ability to learn an algorithm without the benefit of complete information about it. We consider the properties of networks and algorithms that determine the efficiency of generalization. These properties are described in quantitative terms. The most effective generalization is shown to be achieved by networks with the least admissible capacity. General conclusions are illustrated by computer simulations for a three-layered neural network. We draw a quantitative comparison between the general equations and specific results reported here and elsewhere.     

The ability of neuronal networks to generalize using an induction method

Inductive generalization means the ability of a neural network to learn a given algorithm using incomplete information about it. A consideration based on the information theory leads to a simple equation connecting characteristics of the network with those of the algorithm to be learned. The main conclusion is that the most efficient generalization is achieved on the networks with minimal complexity sufficient for realization of the algorithm under consideration. The obtained equation is compared with the results of computer simulations for a universal neural network obtained in the present paper as well as by other workers. A good agreement is observed between theoretical predictions for generalization efficacy and results of computer simulations.     

Informational properties of neural nets performing algorithmic and logical tasks

It is argued that the genetic information necessary to encode an algorithmic neural processor tutoring an otherwise randomly connected biological neural net is represented by the entropy of the analogous minimal Turing machine. Such a near-minimal machine is constructed performing the whole range of bivalent propositional logic in variables. Neural nets computing the same task are presented; their informational entropy can be gauged with reference to the analogous Turing machine. It is also shown that nets with one hidden layer can be trained to perform algorithms solving propositional logic by error back-propagation. Received: 30 June 1995 / Accepted in revised form: 9 January 1996     

Dispersal of aphids, whiteflies and their natural enemies under photoselective nets

Integrated Pest Management of insects includes several control tactics, such as the use of photoselective nets, which may reduce the flight activity of insects. Limiting the dispersal of pests such as aphids and whiteflies is important because of their major role as vectors of plant viruses, while a minor impact on natural enemies is desired. In this study, we examined for the first time the dispersal ability of three vector species, Bemisia tabaci (Gennadius) (Hemiptera: Aleyrodidae), Macrosiphum euphorbiae (Thomas) (Hemiptera: Aphididae) and Myzus persicae (Sulzer) (Hemiptera: Aphididae), in cages covered with photoselective nets. Contrary to the results obtained with aphids, the ability of the whitefly B. tabaci, to reach the target plant was reduced by photoselective nets. In a second set of experiments, the impact of UV-absorbing nets on the visual cues of two important predator species, Orius laevigatus (Fieber) (Hemiptera: Anthocoridae) and Amblyseius swirskii Athias-Henriot (Acari: Phytoseiidae), was evaluated. The anthocorid was caught in higher numbers in traps placed under regular nets, whereas the mites preferably chose environments in which the UV radiation was attenuated. We have observed a wide range of effects that impedes generalization, although photoselective nets have a positive effect on pest management of whiteflies and aphids under protected environments.     

Artificial neural networks and their use in quantitative pathology

A brief general introduction to artificial neural networks is presented, examining in detail the structure and operation of a prototype net developed for the solution of a simple pattern recognition problem in quantitative pathology. The process by which a neural network learns through example and gradually embodies its knowledge as a distributed representation is discussed, using this example. The application of neurocomputer technology to problems in quantitative pathology is explored, using real-world and illustrative examples. Included are examples of the use of artificial neural networks for pattern recognition, database analysis and machine vision. In the context of these examples, characteristics of neural nets, such as their ability to tolerate ambiguous, noisy and spurious data and spontaneously generalize from known examples to handle unfamiliar cases, are examined. Finally, the strengths and deficiencies of a connectionist approach are compared to those of traditional symbolic expert system methodology. It is concluded that artificial neural networks, used in conjunction with other nonalgorithmic artificial intelligence techniques and traditional algorithmic processing, may provide useful software engineering tools for the development of systems in quantitative pathology.     

A new model for simple neural nets and its application in the design of a neural oscillator

The results of a previous theoretical study of a class of systems are applied for the design of neural nets which try to simulate biological behavior. Besides the models for single aperiodic and periodic neurons, a “neural oscillator” is developed which consists of two cross-excited neurons. Its response is similar to the firing pattern of certain biological neural oscillators, like the flying system of the locust. Also, by proper change of its parameters, it can be made highly irregular, providing a deterministic model for the spontaneous neural activity.