Rule extraction from minimal neural networks for credit card screening

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.     

Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, e.g. fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of RNNs may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics.     

Facial expression (mood) recognition from facial images using committee neural networks

Background  

Facial expressions are important in facilitating human communication and interactions. Also, they are used as an important tool in behavioural studies and in medical rehabilitation. Facial image based mood detection techniques may provide a fast and practical approach for non-invasive mood detection. The purpose of the present study was to develop an intelligent system for facial image based expression classification using committee neural networks.     

The use of the PAPNET automated cytological screening system for the diagnosis of oral squamous carcinoma

levine t. s., njemenze v., cowpe j. g. and coleman d. v. (1998) Cytopathology 9, 398–405
The use of the PAPNET automated cytological screening system for the diagnosis of oral squamous carcinoma
The automated PAPNET screening system has been developed to recognize abnormal cells in cervical smears. Given that the oral mucosa sheds cells resembling superficial and intermediate cells of the cervix, the aim of this study was to assess whether the PAPNET system could be used to detect dysplastic cells in oral mucosal smears. Sixty-two oral smears from 27 patients were examined by both light microscopy and using the PAPNET system from clinically abnormal and normal areas by two pathologists. The clinically abnormal sites were also biopsied for histological analysis. There was 100% correlation between the manual and PAPNET screening results. Cytological interpretation of oral smears by both manual and PAPNET screening methods correctly diagnosed squamous cell carcinoma in 14/23 (61%) of patients who had all been confirmed by biopsy. The nine patients with false-negative cases could be attributed to poor smear technique and preparation. The PAPNET system can be used to identify abnormal cells in oral smears and, as such, may have an application for screening those populations at high risk of oral cancer—provided that adequate tuition is given in smear technique.     

Thrips (Thysanoptera) identification using artificial neural networks

We studied the use of a supervised artificial neural network (ANN) model for semi-automated identification of 18 common European species of Thysanoptera from four genera: Aeolothrips Haliday (Aeolothripidae), Chirothrips Haliday, Dendrothrips Uzel, and Limothrips Haliday (all Thripidae). As input data, we entered 17 continuous morphometric and two qualitative two-state characters measured or determined on different parts of the thrips body (head, pronotum, forewing and ovipositor) and the sex. Our experimental data set included 498 thrips specimens. A relatively simple ANN architecture (multilayer perceptrons with a single hidden layer) enabled a 97% correct simultaneous identification of both males and females of all the 18 species in an independent test. This high reliability of classification is promising for a wider application of ANN in the practice of Thysanoptera identification.     

Leveraging high-throughput screening data,deep neural networks,and conditional generative adversarial networks to advance predictive toxicology

There are currently 85,000 chemicals registered with the Environmental Protection Agency (EPA) under the Toxic Substances Control Act, but only a small fraction have measured toxicological data. To address this gap, high-throughput screening (HTS) and computational methods are vital. As part of one such HTS effort, embryonic zebrafish were used to examine a suite of morphological and mortality endpoints at six concentrations from over 1,000 unique chemicals found in the ToxCast library (phase 1 and 2). We hypothesized that by using a conditional generative adversarial network (cGAN) or deep neural networks (DNN), and leveraging this large set of toxicity data we could efficiently predict toxic outcomes of untested chemicals. Utilizing a novel method in this space, we converted the 3D structural information into a weighted set of points while retaining all information about the structure. In vivo toxicity and chemical data were used to train two neural network generators. The first was a DNN (Go-ZT) while the second utilized cGAN architecture (GAN-ZT) to train generators to produce toxicity data. Our results showed that Go-ZT significantly outperformed the cGAN, support vector machine, random forest and multilayer perceptron models in cross-validation, and when tested against an external test dataset. By combining both Go-ZT and GAN-ZT, our consensus model improved the SE, SP, PPV, and Kappa, to 71.4%, 95.9%, 71.4% and 0.673, respectively, resulting in an area under the receiver operating characteristic (AUROC) of 0.837. Considering their potential use as prescreening tools, these models could provide in vivo toxicity predictions and insight into the hundreds of thousands of untested chemicals to prioritize compounds for HT testing.     

The confusion effect from neural networks to reduced predation risk

The confusion effect is often cited as an antipredatory benefitof group living and has been demonstrated by numerous studiesacross a range of taxa. However, there have been relativelyfew studies examining the mechanisms behind the effect and noexperimental test of its supposed theoretical basis (informationdegradation in neural networks) using a natural predator–preypairing. In agreement with other studies, we demonstrate thatattack success of the three-spined stickleback (Gasterosteusaculeatus L.) is reduced by an increase in Daphnia magna groupsize. Neural network models attempt to explain this trend withmultiple prey inducing poor neural mapping of target prey, thusleading to an increase in the spatial error of each attack.We explicitly tested this prediction and demonstrate that thedecrease in attack success by sticklebacks does correspond toan increase in spatial targeting error with larger prey groupsize. Finally, we show that the number of targets, rather thanthe density or area occupied by the group, has the greatesteffect on reducing the rate of attack. These results are discussedin the context of the information processing constraints ofpredators, the ultimate cause of the confusion effect.     

Drawing inspiration from biological dendrites to empower artificial neural networks

This article highlights specific features of biological neurons and their dendritic trees, whose adoption may help advance artificial neural networks used in various machine learning applications. Advancements could take the form of increased computational capabilities and/or reduced power consumption. Proposed features include dendritic anatomy, dendritic nonlinearities, and compartmentalized plasticity rules, all of which shape learning and information processing in biological networks. We discuss the computational benefits provided by these features in biological neurons and suggest ways to adopt them in artificial neurons in order to exploit the respective benefits in machine learning.     

L(p) approximation capabilities of sum-of-product and sigma-pi-sigma neural networks

This paper studies the L(p) approximation capabilities of sum-of-product (SOPNN) and sigma-pi-sigma (SPSNN) neural networks. It is proved that the set of functions that are generated by the SOPNN with its activation function in $L_{loc};p(\mathcal{R})$ is dense in $L;p(\mathcal{K})$ for any compact set $\mathcal{K}\subset \mathcal{R};N$, if and only if the activation function is not a polynomial almost everywhere. It is also shown that if the activation function of the SPSNN is in ${L_{loc};\infty(\mathcal{R})}$, then the functions generated by the SPSNN are dense in $L;p(\mathcal{K})$ if and only if the activation function is not a constant (a.e.).     

Signal-to-signal neural networks for improved spike estimation from calcium imaging data

Spiking information of individual neurons is essential for functional and behavioral analysis in neuroscience research. Calcium imaging techniques are generally employed to obtain activities of neuronal populations. However, these techniques result in slowly-varying fluorescence signals with low temporal resolution. Estimating the temporal positions of the neuronal action potentials from these signals is a challenging problem. In the literature, several generative model-based and data-driven algorithms have been studied with varied levels of success. This article proposes a neural network-based signal-to-signal conversion approach, where it takes as input raw-fluorescence signal and learns to estimate the spike information in an end-to-end fashion. Theoretically, the proposed approach formulates the spike estimation as a single channel source separation problem with unknown mixing conditions. The source corresponding to the action potentials at a lower resolution is estimated at the output. Experimental studies on the spikefinder challenge dataset show that the proposed signal-to-signal conversion approach significantly outperforms state-of-the-art-methods in terms of Pearson’s correlation coefficient, Spearman’s rank correlation coefficient and yields comparable performance for the area under the receiver operating characteristics measure. We also show that the resulting system: (a) has low complexity with respect to existing supervised approaches and is reproducible; (b) is layer-wise interpretable, and (c) has the capability to generalize across different calcium indicators.     

Rules extraction from neural networks applied to the prediction and recognition of prokaryotic promoters

Promoters are DNA sequences located upstream of the gene region and play a central role in gene expression. Computational techniques show good accuracy in gene prediction but are less successful in predicting promoters, primarily because of the high number of false positives that reflect characteristics of the promoter sequences. Many machine learning methods have been used to address this issue. Neural Networks (NN) have been successfully used in this field because of their ability to recognize imprecise and incomplete patterns characteristic of promoter sequences. In this paper, NN was used to predict and recognize promoter sequences in two data sets: (i) one based on nucleotide sequence information and (ii) another based on stability sequence information. The accuracy was approximately 80% for simulation (i) and 68% for simulation (ii). In the rules extracted, biological consensus motifs were important parts of the NN learning process in both simulations.     

Artificial neural networks can learn to estimate extinction rates from molecular phylogenies

Molecular phylogenies typically consist of only extant species, yet they allow inference of past rates of extinction, because recently originated species are less likely to be extinct than ancient species. Despite the simple structure of the assumed underlying speciation-extinction process, parametric functions to estimate extinction rates from phylogenies turned out to be complex and often difficult to derive. Moreover, these parametric functions are specific to a particular process (e.g. complete species level phylogeny with constant birth and death rates) and a particular type of data (e.g. times between bifurcations). Here, it is shown that artificial neural networks can substitute for parametric estimation functions once they have been sufficiently trained on simulated data. This technique can in principle be used for different processes and data types, and because it circumvents the time-consuming and difficult task of deriving parametric estimation functions, it may greatly extend the possibilities to make macro-evolutionary inferences from molecular phylogenies. This novel approach is explained, applied to estimate speciation and extinction rates from a molecular phylogeny of the reef fish genus Naso (Acanturidae), and its performance is compared to that of maximum likelihood estimation.     

Mathematical model for the self-organization of neural networks

Mutual inhibition between neurons combined with a learning principle similar to that proposed by Hebb is shown to secure a powerful selforganizing property for neural networks. Numerical analysis reveals that the system investigated always organizes itself into the same final state from any arbitrarily chosen initial state.     

Spatial constancy and the brain: insights from neural networks

To form an accurate internal representation of visual space, the brain must accurately account for movements of the eyes, head or body. Updating of internal representations in response to these movements is especially important when remembering spatial information, such as the location of an object, since the brain must rely on non-visual extra-retinal signals to compensate for self-generated movements. We investigated the computations underlying spatial updating by constructing a recurrent neural network model to store and update a spatial location based on a gaze shift signal, and to do so flexibly based on a contextual cue. We observed a striking similarity between the patterns of behaviour produced by the model and monkeys trained to perform the same task, as well as between the hidden units of the model and neurons in the lateral intraparietal area (LIP). In this report, we describe the similarities between the model and single unit physiology to illustrate the usefulness of neural networks as a tool for understanding specific computations performed by the brain.     

Cellular neural networks (CNN) with linear weight functions for a prediction of epileptic seizures

In this paper, we present a novel approach to the prediction of epileptic seizures using boolean CNN with linear weight functions. Three different binary pattern occurrence behaviours will be discussed and analysed for several invasive recordings of brain electrical activity. Furthermore analogic binary pattern detection algorithms will be introduced for a possible prediction of epileptic seizures.     

Aspects on the integrative actions of the brain from neural networks to "brain-body medicine"

"Integration" is a key term in describing how nervous system can perform high level functions. A first condition to have "integration" is obviously the presence of efficient "communication processes" among the parts that have to be combined into the harmonious whole. In this respect, two types of communication processes, called wiring transmission (WT) and volume transmission (VT), respectively, were found to play a major role in the nervous system, allowing the exchange of signals not only between neurons, but rather among all cell types present in the central nervous system (CNS). A second fundamental aspect of a communication process is obviously the recognition/decoding process at target level. As far as this point is concerned, increasing evidence emphasizes the importance of supramolecular complexes of receptors (the so called receptor mosaics) generated by direct receptor-receptor interactions. Their assemblage would allow a first integration of the incoming information already at the plasma membrane level. Recently, evidence of two new subtypes of WT and VT has been obtained, namely the tunnelling nanotubes mediated WT and the microvesicle (in particular exosomes) mediated VT allowing the horizontal transfer of bioactive molecules, including receptors, RNAs and micro-RNAs. The physiological and pathological implications of these types of communication have opened up a new field that is largely still unexplored. In fact, likely unsuspected integrative actions of the nervous system could occur. In this context, a holistic approach to the brain-body complex as an indissoluble system has been proposed. Thus, the hypothesis has been introduced on the existence of a brain-body integrative structure formed by the "area postrema/nucleus tractus solitarius" (AP/NTS) and the "anteroventral third ventricle region/basal hypothalamus with the median eminence" (AV3V-BH). These highly interconnected regions operate as specialized interfaces between the brain and the body integrating brain-borne and body-borne neural and humoral signals.     

Neuropathological findings processed by artificial neural networks (ANNs) can perfectly distinguish Alzheimer's patients from controls in the Nun Study

Background  

Many reports have described that there are fewer differences in AD brain neuropathologic lesions between AD patients and control subjects aged 80 years and older, as compared with the considerable differences between younger persons with AD and controls. In fact some investigators have suggested that since neurofibrillary tangles (NFT) can be identified in the brains of non-demented elderly subjects they should be considered as a consequence of the aging process. At present, there are no universally accepted neuropathological criteria which can mathematically differentiate AD from healthy brain in the oldest old.     

Voltage-stepping schemes for the simulation of spiking neural networks

The numerical simulation of spiking neural networks requires particular attention. On the one hand, time-stepping methods are generic but they are prone to numerical errors and need specific treatments to deal with the discontinuities of integrate-and-fire models. On the other hand, event-driven methods are more precise but they are restricted to a limited class of neuron models. We present here a voltage-stepping scheme that combines the advantages of these two approaches and consists of a discretization of the voltage state-space. The numerical simulation is reduced to a local event-driven method that induces an implicit activity-dependent time discretization (time-steps automatically increase when the neuron is slowly varying). We show analytically that such a scheme leads to a high-order algorithm so that it accurately approximates the neuronal dynamics. The voltage-stepping method is generic and can be used to simulate any kind of neuron models. We illustrate it on nonlinear integrate-and-fire models and show that it outperforms time-stepping schemes of Runge-Kutta type in terms of simulation time and accuracy.

Using the antibody-antigen binding interface to train image-based deep neural networks for antibody-epitope classification

High-throughput B-cell sequencing has opened up new avenues for investigating complex mechanisms underlying our adaptive immune response. These technological advances drive data generation and the need to mine and analyze the information contained in these large datasets, in particular the identification of therapeutic antibodies (Abs) or those associated with disease exposure and protection. Here, we describe our efforts to use artificial intelligence (AI)-based image-analyses for prospective classification of Abs based solely on sequence information. We hypothesized that Abs recognizing the same part of an antigen share a limited set of features at the binding interface, and that the binding site regions of these Abs share share common structure and physicochemical property patterns that can serve as a “fingerprint” to recognize uncharacterized Abs. We combined large-scale sequence-based protein-structure predictions to generate ensembles of 3-D Ab models, reduced the Ab binding interface to a 2-D image (fingerprint), used pre-trained convolutional neural networks to extract features, and trained deep neural networks (DNNs) to classify Abs. We evaluated this approach using Ab sequences derived from human HIV and Ebola viral infections to differentiate between two Abs, Abs belonging to specific B-cell family lineages, and Abs with different epitope preferences. In addition, we explored a different type of DNN method to detect one class of Abs from a larger pool of Abs. Testing on Ab sets that had been kept aside during model training, we achieved average prediction accuracies ranging from 71–96% depending on the complexity of the classification task. The high level of accuracies reached during these classification tests suggests that the DNN models were able to learn a series of structural patterns shared by Abs belonging to the same class. The developed methodology provides a means to apply AI-based image recognition techniques to analyze high-throughput B-cell sequencing datasets (repertoires) for Ab classification.