A global competitive neural network

 A study is presented of a set of coupled nets proposed to function as a global competitive network. One net, of hidden nodes, is composed solely of inhibitory neurons and is excitatorily driven and feeds back in a disinhibitory manner to an input net which itself feeds excitatorily to a (cortical) output net. The manner in which the former input and hidden inhibitory net function so as to enhance outputs as compared with inputs, and the further enhancements when the cortical net is added, are explored both mathematically and by simulation. This is extended to learning on cortical afferent and lateral connections. A global wave structure, arising on the inhibitory net in a similar manner to that of pattern formation in a negative laplacian net, is seen to be important to all of these activities. Simulations are only performed in one dimension, although the global nature of the activity is expected to extend to higher dimensions. Possible implications are briefly discussed. Received: 21 November 1993/Accepted in revised form: 30 June 1994     

Familiarity recognition and recollection: A neural network model

The capacities of a specially designed neural network for familiarity recognition and recollection have been compared. Recognition is based on calculating “image familiarity” as a modified Hopfield energy function in which the value of the inner sum is replaced by the sign of this value. This replacement makes the calculation of familiarity compatible with the basic dynamic equations of the Hopfield network and is in fact reduced to calculating the scalar product of the neuronet state vectors at two successive time steps.     

A new neural framework for visuospatial processing

The division of cortical visual processing into distinct dorsal and ventral streams is a key framework that has guided visual neuroscience. The characterization of the ventral stream as a 'What' pathway is relatively uncontroversial, but the nature of dorsal stream processing is less clear. Originally proposed as mediating spatial perception ('Where'), more recent accounts suggest it primarily serves non-conscious visually guided action ('How'). Here, we identify three pathways emerging from the dorsal stream that consist of projections to the prefrontal and premotor cortices, and a major projection to the medial temporal lobe that courses both directly and indirectly through the posterior cingulate and retrosplenial cortices. These three pathways support both conscious and non-conscious visuospatial processing, including spatial working memory, visually guided action and navigation, respectively.     

Disease named entity recognition from biomedical literature using a novel convolutional neural network

Background

Automatic disease named entity recognition (DNER) is of utmost importance for development of more sophisticated BioNLP tools. However, most conventional CRF based DNER systems rely on well-designed features whose selection is labor intensive and time-consuming. Though most deep learning methods can solve NER problems with little feature engineering, they employ additional CRF layer to capture the correlation information between labels in neighborhoods which makes them much complicated.

Methods

In this paper, we propose a novel multiple label convolutional neural network (MCNN) based disease NER approach. In this approach, instead of the CRF layer, a multiple label strategy (MLS) first introduced by us, is employed. First, the character-level embedding, word-level embedding and lexicon feature embedding are concatenated. Then several convolutional layers are stacked over the concatenated embedding. Finally, MLS strategy is applied to the output layer to capture the correlation information between neighboring labels.

Results

As shown by the experimental results, MCNN can achieve the state-of-the-art performance on both NCBI and CDR corpora.

Conclusions

The proposed MCNN based disease NER method achieves the state-of-the-art performance with little feature engineering. And the experimental results show the MLS strategy’s effectiveness of capturing the correlation information between labels in the neighborhood.

     

Efficient recognition of immunoglobulin domains from amino acid sequences using a neural network

A neural network was trained using back propagation to recognizeimmunoglobulin domains from amino acid sequences. The programwas designed to identify proteins exhibiting such domains withminimal rates of false positives and false negatives. The NationalBiomedical Research Foundation NEW protein sequences databasewas scanned to evaluate the performance of the program in recognizingmouse immunoglobulin sequences. The program correctly recognized55 out of 56 mouse immunoglobulin sequences, corresponding toa recognition efficiency of 98.2% with an overall false positiverate of 7.3%. These data demonstrate that neural network-basedsearch programs are well suited to search for sequences characterizedby only a few well-conserved subsequences. Received on November 11, 1989; accepted on May 31, 1990     

A neural network for processing olfactory-like stimuli

Several critical issues associated with the processing of olfactory stimuli in animals (but focusing on insects) are discussed with a view to designing a neural network which can process olfactory stimuli. This leads to the construction of a neural network that can learn and identify the quality (direction cosines) of an input vector or extract information from a sequence of correlated input vectors, where the latter corresponds to sampling a time varying olfactory stimulus (or other generically similar pattern recognition problems). The network is constructed around a discrete time content-addressable memory (CAM) module which basically satisfies the Hopfield equations with the addition of a unit time delay feedback. This modification improves the convergence properties of the network and is used to control a switch which activates the learning or template formation process when the input is “unknown”. The network dynamics are embedded within a sniff cycle which includes a larger time delay (i.e. an integert _s <1) that is also used to control the template formation switch. In addition, this time delay is used to modify the input into the CAM module so that the more dominant of two mingling odors or an odor increasing against a background of odors is more readily identified. The performance of the network is evaluated using Monte Carlo simulations and numerical results are presented.     

Application of competitive Hopfield neural network to brain-computer interface systems

We propose an unsupervised recognition system for single-trial classification of motor imagery (MI) electroencephalogram (EEG) data in this study. Competitive Hopfield neural network (CHNN) clustering is used for the discrimination of left and right MI EEG data posterior to selecting active segment and extracting fractal features in multi-scale. First, we use continuous wavelet transform (CWT) and Student's two-sample t-statistics to select the active segment in the time-frequency domain. The multiresolution fractal features are then extracted from wavelet data by means of modified fractal dimension. At last, CHNN clustering is adopted to recognize extracted features. Due to the characteristic of non-supervision, it is proper for CHNN to classify non-stationary EEG signals. The results indicate that CHNN achieves 81.9% in average classification accuracy in comparison with self-organizing map (SOM) and several popular supervised classifiers on six subjects from two data sets.     

Speech recognition by an artificial neural network using findings on the afferent auditory system

An artificial neural network which uses anatomical and physiological findings on the afferent pathway from the ear to the cortex is presented and the roles of the constituent functions in recognition of continuous speech are examined. The network deals with successive spectra of speech sounds by a cascade of several neural layers: lateral excitation layer (LEL), lateral inhibition layer (LIL), and a pile of feature detection layers (FDL's). These layers are shown to be effective for recognizing spoken words. Namely, first, LEL reduces the distortion of sound spectrum caused by the pitch of speech sounds. Next, LIL emphasizes the major energy peaks of sound spectrum, the formants. Last, FDL's detect syllables and words in successive formants, where two functions, time-delay and strong adaptation, play important roles: time-delay makes it possible to retain the pattern of formant changes for a period to detect spoken words successively; strong adaptation contributes to removing the time-warp of formant changes. Digital computer simulations show that the network detect isolated syllables, isolated words, and connected words in continuous speech, while reproducing the fundamental responses found in the auditory system such as ON, OFF, ON-OFF, and SUSTAINED patterns.     

Oscillatory neural network for pattern recognition: trajectory based classification and supervised learning

Computer algorithms that match human performance in recognizing written text or spoken conversation remain elusive. The reasons why the human brain far exceeds any existing recognition scheme to date in the ability to generalize and to extract invariant characteristics relevant to category matching are not clear. However, it has been postulated that the dynamic distribution of brain activity (spatiotemporal activation patterns) is the mechanism by which stimuli are encoded and matched to categories. This research focuses on supervised learning using a trajectory based distance metric for category discrimination in an oscillatory neural network model. Classification is accomplished using a trajectory based distance metric. Since the distance metric is differentiable, a supervised learning algorithm based on gradient descent is demonstrated. Classification of spatiotemporal frequency transitions and their relation to a priori assessed categories is shown along with the improved classification results after supervised training. The results indicate that this spatiotemporal representation of stimuli and the associated distance metric is useful for simple pattern recognition tasks and that supervised learning improves classification results.     

Introspective minds: using ALE meta-analyses to study commonalities in the neural correlates of emotional processing, social & unconstrained cognition

Previous research suggests overlap between brain regions that show task-induced deactivations and those activated during the performance of social-cognitive tasks. Here, we present results of quantitative meta-analyses of neuroimaging studies, which confirm a statistical convergence in the neural correlates of social and resting state cognition. Based on the idea that both social and unconstrained cognition might be characterized by introspective processes, which are also thought to be highly relevant for emotional experiences, a third meta-analysis was performed investigating studies on emotional processing. By using conjunction analyses across all three sets of studies, we can demonstrate significant overlap of task-related signal change in dorso-medial prefrontal and medial parietal cortex, brain regions that have, indeed, recently been linked to introspective abilities. Our findings, therefore, provide evidence for the existence of a core neural network, which shows task-related signal change during socio-emotional tasks and during resting states.     

A model of neural network for spatiotemporal pattern recognition

A model of neural network to recognize spatiotemporal patterns is presented. The network consists of two kinds of neural cells: P-cells and B-cells. A P-cell generates an impulse responding to more than one impulse and embodies two special functions: short term storage (STS) and heterosynaptic facilitation (HSF). A B-cell generates several impulses with high frequency as soon as it receives an impulse. In recognizing process, an impulse generated by a P-cell represents a recognition of stimulus pattern, and triggers the generation of impulses of a B-cell. Inhibitory impulses with high frequency generated by a B-cell reset the activities of all P-cells in the network.Two examples of spatiotemporal pattern recognition are presented. They are achieved by giving different values to the parameters of the network. In one example, the network recognizes both directional and non-directional patterns. The selectivities to directional and non-directional patterns are realized by only adjusting excitatory synaptic weights of P-cells. In the other example, the network recognizes time series of spatial patterns, where the lengths of the series are not necessarily the same and the transitional speeds of spatial patterns are not always the same. In both examples, the HSF signal controls the total activity of the network, which contributes to exact recognition and error recovery. In the latter example, it plays a role to trigger and execute the recognizing process. Finally, we discuss the correspondence between the model and physiological findings.     

A neural network approach to evaluate fold recognition results

Fold recognition techniques assist the exploration of protein structures, and web-based servers are part of the standard set of tools used in the analysis of biochemical problems. Despite their success, current methods are only able to predict the correct fold in a relatively small number of cases. We propose an approach that improves the selection of correct folds from among the results of two methods implemented as web servers (SAMT99 and 3DPSSM). Our approach is based on the training of a system of neural networks with models generated by the servers and a set of associated characteristics such as the quality of the sequence-structure alignment, distribution of sequence features (sequence-conserved positions and apolar residues), and compactness of the resulting models. Our results show that it is possible to detect adequate folds to model 80% of the sequences with a high level of confidence. The improvements achieved by taking into account sequence characteristics open the door to future improvements by directly including such factors in the step of model generation. This approach has been implemented as an automatic system LIBELLULA, available as a public web server at http://www.pdg.cnb.uam.es/servers/libellula.html.     

ProDCoNN: Protein design using a convolutional neural network

Designing protein sequences that fold to a given three-dimensional (3D) structure has long been a challenging problem in computational structural biology with significant theoretical and practical implications. In this study, we first formulated this problem as predicting the residue type given the 3D structural environment around the C _α atom of a residue, which is repeated for each residue of a protein. We designed a nine-layer 3D deep convolutional neural network (CNN) that takes as input a gridded box with the atomic coordinates and types around a residue. Several CNN layers were designed to capture structure information at different scales, such as bond lengths, bond angles, torsion angles, and secondary structures. Trained on a very large number of protein structures, the method, called ProDCoNN (protein design with CNN), achieved state-of-the-art performance when tested on large numbers of test proteins and benchmark datasets.     

Image segmentation using a neural network

An object extraction problem based on the Gibbs Random Field model is discussed. The Maximum a'posteriori probability (MAP) estimate of a scene based on a noise-corrupted realization is found to be computationally exponential in nature. A neural network, which is a modified version of that of Hopfield, is suggested for solving the problem. A single neuron is assigned to every pixel. Each neuron is supposed to be connected only to all of its nearest neighbours. The energy function of the network is designed in such a way that its minimum value corresponds to the MAP estimate of the scene. The dynamics of the network are described. A possible hardware realization of a neuron is also suggested. The technique is implemented on a set of noisy images and found to be highly robust and immune to noise.     

A hypothesis-testing framework for studies investigating ontogenetic niche shifts using stable isotope ratios

Ontogenetic niche shifts occur across diverse taxonomic groups, and can have critical implications for population dynamics, community structure, and ecosystem function. In this study, we provide a hypothesis-testing framework combining univariate and multivariate analyses to examine ontogenetic niche shifts using stable isotope ratios. This framework is based on three distinct ontogenetic niche shift scenarios, i.e., (1) no niche shift, (2) niche expansion/reduction, and (3) discrete niche shift between size classes. We developed criteria for identifying each scenario, as based on three important resource use characteristics, i.e., niche width, niche position, and niche overlap. We provide an empirical example for each ontogenetic niche shift scenario, illustrating differences in resource use characteristics among different organisms. The present framework provides a foundation for future studies on ontogenetic niche shifts, and also can be applied to examine resource variability among other population sub-groupings (e.g., by sex or phenotype).     

Sentence alignment using feed forward neural network

Parallel corpora have become an essential resource for work in multi lingual natural language processing. However, sentence aligned parallel corpora are more efficient than non-aligned parallel corpora for cross language information retrieval and machine translation applications. In this paper, we present a new approach to align sentences in bilingual parallel corpora based on feed forward neural network classifier. A feature parameter vector is extracted from the text pair under consideration. This vector contains text features such as length, punctuate score, and cognate score values. A set of manually prepared training data has been assigned to train the feed forward neural network. Another set of data was used for testing. Using this new approach, we could achieve an error reduction of 60% over length based approach when applied on English-Arabic parallel documents. Moreover this new approach is valid for any language pair and it is quite flexible approach since the feature parameter vector may contain more/less or different features than that we used in our system such as lexical match feature.     

A neural network model for selective attention in visual pattern recognition

A neural network model of the mechanism of selective attention in visual pattern recognition is proposed and simulated on a digital computer.When a complex figure consisting of two patterns or more is presented to the model, it is segmented into individual patterns, and each pattern is recognized separately. Even if one of the patterns to which the model is paying selective attention is affected by noise or defects, the model can recall the complete pattern from which the noise has been eliminated and the defects corrected. It is not necessary for perfect recall that the stimulus pattern should be identical in shape to the training pattern. Even though the pattern is distorted in shape or changed in size, it can be correctly recognized and the missing portions restored.The model consists of a hierarchical neural network which has efferent as well as afferent connections between cells. The afferent and the efferent signals interact with each other in the network: the efferent signals, that is, the signals for selective attention, have a facilitating effect on the afferent ones, and, at the same time, the afferent signals gate efferent signal flow. When some feature in the stimulus is not extracted in the afferent paths, the threshold for detection of that feature is automatically lowered by decreasing the efficiency of inhibition, and the model tries to extract even vague traces of the undetected feature.     

Modeling fluctuations in default-mode brain network using a spiking neural network

Recently, numerous attempts have been made to understand the dynamic behavior of complex brain systems using neural network models. The fluctuations in blood-oxygen-level-dependent (BOLD) brain signals at less than 0.1 Hz have been observed by functional magnetic resonance imaging (fMRI) for subjects in a resting state. This phenomenon is referred to as a "default-mode brain network." In this study, we model the default-mode brain network by functionally connecting neural communities composed of spiking neurons in a complex network. Through computational simulations of the model, including transmission delays and complex connectivity, the network dynamics of the neural system and its behavior are discussed. The results show that the power spectrum of the modeled fluctuations in the neuron firing patterns is consistent with the default-mode brain network's BOLD signals when transmission delays, a characteristic property of the brain, have finite values in a given range.