A combinational feature selection and ensemble neural network method for classification of gene expression data

Background  

Microarray experiments are becoming a powerful tool for clinical diagnosis, as they have the potential to discover gene expression patterns that are characteristic for a particular disease. To date, this problem has received most attention in the context of cancer research, especially in tumor classification. Various feature selection methods and classifier design strategies also have been generally used and compared. However, most published articles on tumor classification have applied a certain technique to a certain dataset, and recently several researchers compared these techniques based on several public datasets. But, it has been verified that differently selected features reflect different aspects of the dataset and some selected features can obtain better solutions on some certain problems. At the same time, faced with a large amount of microarray data with little knowledge, it is difficult to find the intrinsic characteristics using traditional methods. In this paper, we attempt to introduce a combinational feature selection method in conjunction with ensemble neural networks to generally improve the accuracy and robustness of sample classification.     

A probabilistic neural network approach for modeling and classification of bacterial growth/no-growth data

In this paper, we propose to use probabilistic neural networks (PNNs) for classification of bacterial growth/no-growth data and modeling the probability of growth. The PNN approach combines both Bayes theorem of conditional probability and Parzen's method for estimating the probability density functions of the random variables. Unlike other neural network training paradigms, PNNs are characterized by high training speed and their ability to produce confidence levels for their classification decision. As a practical application of the proposed approach, PNNs were investigated for their ability in classification of growth/no-growth state of a pathogenic Escherichia coli R31 in response to temperature and water activity. A comparison with the most frequently used traditional statistical method based on logistic regression and multilayer feedforward artificial neural network (MFANN) trained by error backpropagation was also carried out. The PNN-based models were found to outperform linear and nonlinear logistic regression and MFANN in both the classification accuracy and ease by which PNN-based models are developed.     

A Bayesian network classification methodology for gene expression data.

We present new techniques for the application of a Bayesian network learning framework to the problem of classifying gene expression data. The focus on classification permits us to develop techniques that address in several ways the complexities of learning Bayesian nets. Our classification model reduces the Bayesian network learning problem to the problem of learning multiple subnetworks, each consisting of a class label node and its set of parent genes. We argue that this classification model is more appropriate for the gene expression domain than are other structurally similar Bayesian network classification models, such as Naive Bayes and Tree Augmented Naive Bayes (TAN), because our model is consistent with prior domain experience suggesting that a relatively small number of genes, taken in different combinations, is required to predict most clinical classes of interest. Within this framework, we consider two different approaches to identifying parent sets which are supported by the gene expression observations and any other currently available evidence. One approach employs a simple greedy algorithm to search the universe of all genes; the second approach develops and applies a gene selection algorithm whose results are incorporated as a prior to enable an exhaustive search for parent sets over a restricted universe of genes. Two other significant contributions are the construction of classifiers from multiple, competing Bayesian network hypotheses and algorithmic methods for normalizing and binning gene expression data in the absence of prior expert knowledge. Our classifiers are developed under a cross validation regimen and then validated on corresponding out-of-sample test sets. The classifiers attain a classification rate in excess of 90% on out-of-sample test sets for two publicly available datasets. We present an extensive compilation of results reported in the literature for other classification methods run against these same two datasets. Our results are comparable to, or better than, any we have found reported for these two sets, when a train-test protocol as stringent as ours is followed.     

Novel neural network application for bacterial colony classification

Background

Bacterial colony morphology is the first step of classifying the bacterial species before sending them to subsequent identification process with devices, such as VITEK 2 automated system and mass spectrometry microbial identification system. It is essential as a pre-screening process because it can greatly reduce the scope of possible bacterial species and will make the subsequent identification more specific and increase work efficiency in clinical bacteriology. But this work needs adequate clinical laboratory expertise of bacterial colony morphology, which is especially difficult for beginners to handle properly. This study presents automatic programs for bacterial colony classification task, by applying the deep convolutional neural networks (CNN), which has a widespread use of digital imaging data analysis in hospitals. The most common 18 bacterial colony classes from Peking University First Hospital were used to train this framework, and other images out of these training dataset were utilized to test the performance of this classifier.

Results

The feasibility of this framework was verified by the comparison between predicted result and standard bacterial category. The classification accuracy of all 18 bacteria can reach 73%, and the accuracy and specificity of each kind of bacteria can reach as high as 90%.

Conclusions

The supervised neural networks we use can have more promising classification characteristics for bacterial colony pre-screening process, and the unsupervised network should have more advantages in revealing novel characteristics from pictures, which can provide some practical indications to our clinical staffs.

     

A neural network model with feature selection for Korean speech act classification

A speech act is a linguistic action intended by a speaker. Speech act classification is an essential part of a dialogue understanding system because the speech act of an utterance is closely tied with the user's intention in the utterance. We propose a neural network model for Korean speech act classification. In addition, we propose a method that extracts morphological features from surface utterances and selects effective ones among the morphological features. Using the feature selection method, the proposed neural network can partially increase precision and decrease training time. In the experiment, the proposed neural network showed better results than other models using comparatively high-level linguistic features. Based on the experimental result, we believe that the proposed neural network model is suitable for real field applications because it is easy to expand the neural network model into other domains. Moreover, we found that neural networks can be useful in speech act classification if we can convert surface sentences into vectors with fixed dimensions by using an effective feature selection method.     

FID-Net: A versatile deep neural network architecture for NMR spectral reconstruction and virtual decoupling

In recent years, the transformative potential of deep neural networks (DNNs) for analysing and interpreting NMR data has clearly been recognised. However, most applications of DNNs in NMR to date either struggle to outperform existing methodologies or are limited in scope to a narrow range of data that closely resemble the data that the network was trained on. These limitations have prevented a widescale uptake of DNNs in NMR. Addressing this, we introduce FID-Net, a deep neural network architecture inspired by WaveNet, for performing analyses on time domain NMR data. We first demonstrate the effectiveness of this architecture in reconstructing non-uniformly sampled (NUS) biomolecular NMR spectra. It is shown that a single network is able to reconstruct a diverse range of 2D NUS spectra that have been obtained with arbitrary sampling schedules, with a range of sweep widths, and a variety of other acquisition parameters. The performance of the trained FID-Net in this case exceeds or matches existing methods currently used for the reconstruction of NUS NMR spectra. Secondly, we present a network based on the FID-Net architecture that can efficiently virtually decouple ¹³C_α-¹³C_β couplings in HNCA protein NMR spectra in a single shot analysis, while at the same time leaving glycine residues unmodulated. The ability for these DNNs to work effectively in a wide range of scenarios, without retraining, paves the way for their widespread usage in analysing NMR data.

     

Application of artificial neural network for classification of thyroid follicular tumors

OBJECTIVE: To analyze smears of 197 thyroid follicular tumors (adenoma and carcinoma). STUDY DESIGN: Several types of artificial neural networks (ANN) of various designs were used for diagnosis of thyroid follicular tumors. The typical complex of cytologic features, some nuclear morphometric parameters (area, perimeter, shape factor) and density features of chromatin texture (mean value and SD of gray levels) were defined for each tumor. RESULTS: The ANN was trained by means of cytologic features characteristic for a thyroid follicular adenoma and a follicular carcinoma. At subsequent testing, the correct cytologic diagnosis was established in 93% (25 of 27) of cases. The morphometry increased the accuracy of diagnosis for follicular tumors in up to 97% (75 of 78) of cases. ANN correctly distinguished an adenoma or a carcinoma in 87% (73 of 84) of cases when using color microscopic images of tumors. CONCLUSION: The usage of ANN has raised sensitivity of cytologic diagnosis of follicular tumors to 90%, compared with a usual cytologic method (sensitivity of 56%). The automatic classification of thyroid follicular tumors by means of ANN is prospective.     

A new neural network architecture with associative memory, pruning and order-sensitive learning

A new paradigm of neural network architecture is proposed that works as associative memory along with capabilities of pruning and order-sensitive learning. The network has a composite structure wherein each node of the network is a Hopfield network by itself. The Hopfield network employs an order-sensitive learning technique and converges to user-specified stable states without having any spurious states. This is based on geometrical structure of the network and of the energy function. The network is so designed that it allows pruning in binary order as it progressively carries out associative memory retrieval. The capacity of the network is 2n, where n is the number of basic nodes in the network. The capabilities of the network are demonstrated by experimenting on three different application areas, namely a Library Database, a Protein Structure Database and Natural Language Understanding.     

A system architecture for genomic data analysis

MOTIVATION: Most of diseases are caused by a set of gene defects, which occur in a complex association. The association scheme of expressed genes can be modelled by genetic networks. Genetic networks are efficiently facilities to understand the dynamic of pathogenic processes by modelling molecular reality of cell conditions. In this sense a genetic network consists of first, a set of genes of specified cells, tissues or species and second, causal relations between these genes determining the functional condition of the biological system, i. e. under disease. A relation between two genes will exist if they both are directly or indirectly associated with disease [8]. Our goal is to characterize diseases (especially autoimmune diseases like chronic pancreatitis CP, multiple sclerosis MS, rheumatoid arthritis RA) by genetic networks generated by a computer system. We want to introduce this practice as a bioinformatic approach for finding targets.     

Artificial neural network classification of Drosophila courtship song mutants

Courtship songs produced by Drosophila males — wild-type, plus the cacophony and dissonance behavioral mutants — were examined with the aid of newly developed strategies for adaptive acoustic analysis and classification. This system used several techniques involving artificial neural networks (a.k.a. parallel distributed processing), including learned vector quantization of signals and non-linear adaption (back-propagation) of data analysis. Pulse song from several individual wild-type and mutant males were first vector-quantized according to their frequency spectra. The accumulated quantized data of this kind, for a given song, were then used to teach or adapt a multiple-layered feedforward artificial neural network, which classified that song according to its original genotype. Results are presented on the performance of the final adapted system when faced with novel test data and on acoustic features the system decides upon for predicting the song-mutant genotype in question. The potential applications and extensions of this new system are discussed, including how it could be used to screen for courtship mutants, search novel behavior patterns or cause-and-effect relationships associated with reproduction, compress these kinds of data for digital storage, and analyze Drosophila behavior beyond the case of courtship song.     

A bayesian recurrent neural network for unsupervised pattern recognition in large incomplete data sets

A recurrent neural network, modified to handle highly incomplete training data is described. Unsupervised pattern recognition is demonstrated in the WHO database of adverse drug reactions. Comparison is made to a well established method, AutoClass, and the performances of both methods is investigated on simulated data. The neural network method performs comparably to AutoClass in simulated data, and better than AutoClass in real world data. With its better scaling properties, the neural network is a promising tool for unsupervised pattern recognition in huge databases of incomplete observations.     

A neural network for processing olfactory-like stimuli

Several critical issues associated with the processing of olfactory stimuli in animals (but focusing on insects) are discussed with a view to designing a neural network which can process olfactory stimuli. This leads to the construction of a neural network that can learn and identify the quality (direction cosines) of an input vector or extract information from a sequence of correlated input vectors, where the latter corresponds to sampling a time varying olfactory stimulus (or other generically similar pattern recognition problems). The network is constructed around a discrete time content-addressable memory (CAM) module which basically satisfies the Hopfield equations with the addition of a unit time delay feedback. This modification improves the convergence properties of the network and is used to control a switch which activates the learning or template formation process when the input is “unknown”. The network dynamics are embedded within a sniff cycle which includes a larger time delay (i.e. an integert _s <1) that is also used to control the template formation switch. In addition, this time delay is used to modify the input into the CAM module so that the more dominant of two mingling odors or an odor increasing against a background of odors is more readily identified. The performance of the network is evaluated using Monte Carlo simulations and numerical results are presented.     

A neural network model for trace conditioning

We studied the dynamics of a neural network that has both recurrent excitatory and random inhibitory connections. Neurons started to become active when a relatively weak transient excitatory signal was presented and the activity was sustained due to the recurrent excitatory connections. The sustained activity stopped when a strong transient signal was presented or when neurons were disinhibited. The random inhibitory connections modulated the activity patterns of neurons so that the patterns evolved without recurrence with time. Hence, a time passage between the onsets of the two transient signals was represented by the sequence of activity patterns. We then applied this model to represent the trace eye blink conditioning, which is mediated by the hippocampus. We assumed this model as CA3 of the hippocampus and considered an output neuron corresponding to a neuron in CA1. The activity pattern of the output neuron was similar to that of CA1 neurons during trace eye blink conditioning, which was experimentally observed.     

Prediction of protein secondary structure using improved two-level neural network architecture

In this paper we propose constructing an improved two-level neural network to predict protein secondary structure. Firstly, we code the whole protein composition information as the inputs to the first-level network besides the evolutionary information. Secondly, we calculate the reliability score for each residue position based on the output of the first-level network, and the role of the second-level network is to take full advantage of the residues with a higher reliability score to impact the neighboring residues with a lower one for improving the whole prediction accuracy. Thirdly, considering it is indeed a problem that the target protein can be lost in the multiple sequence alignment we propose to code single sequence into the second-level network. The experimental results show that our proposed method can efficiently improve the prediction accuracy.     

Learning, memory, and the role of neural network architecture

The performance of information processing systems, from artificial neural networks to natural neuronal ensembles, depends heavily on the underlying system architecture. In this study, we compare the performance of parallel and layered network architectures during sequential tasks that require both acquisition and retention of information, thereby identifying tradeoffs between learning and memory processes. During the task of supervised, sequential function approximation, networks produce and adapt representations of external information. Performance is evaluated by statistically analyzing the error in these representations while varying the initial network state, the structure of the external information, and the time given to learn the information. We link performance to complexity in network architecture by characterizing local error landscape curvature. We find that variations in error landscape structure give rise to tradeoffs in performance; these include the ability of the network to maximize accuracy versus minimize inaccuracy and produce specific versus generalizable representations of information. Parallel networks generate smooth error landscapes with deep, narrow minima, enabling them to find highly specific representations given sufficient time. While accurate, however, these representations are difficult to generalize. In contrast, layered networks generate rough error landscapes with a variety of local minima, allowing them to quickly find coarse representations. Although less accurate, these representations are easily adaptable. The presence of measurable performance tradeoffs in both layered and parallel networks has implications for understanding the behavior of a wide variety of natural and artificial learning systems.     

A neural network model for texture discrimination

A model of texture discrimination in visual cortex was built using a feedforward network with lateral interactions among relatively realistic spiking neural elements. The elements have various membrane currents, equilibrium potentials and time constants, with action potentials and synapses. The model is derived from the modified programs of MacGregor (1987). Gabor-like filters are applied to overlapping regions in the original image; the neural network with lateral excitatory and inhibitory interactions then compares and adjusts the Gabor amplitudes in order to produce the actual texture discrimination. Finally, a combination layer selects and groups various representations in the output of the network to form the final transformed image material. We show that both texture segmentation and detection of texture boundaries can be represented in the firing activity of such a network for a wide variety of synthetic to natural images. Performance details depend most strongly on the global balance of strengths of the excitatory and inhibitory lateral interconnections. The spatial distribution of lateral connective strengths has relatively little effect. Detailed temporal firing activities of single elements in the lateral connected network were examined under various stimulus conditions. Results show (as in area 17 of cortex) that a single element's response to image features local to its receptive field can be altered by changes in the global context.     

Oscillatory neural network for pattern recognition: trajectory based classification and supervised learning

Computer algorithms that match human performance in recognizing written text or spoken conversation remain elusive. The reasons why the human brain far exceeds any existing recognition scheme to date in the ability to generalize and to extract invariant characteristics relevant to category matching are not clear. However, it has been postulated that the dynamic distribution of brain activity (spatiotemporal activation patterns) is the mechanism by which stimuli are encoded and matched to categories. This research focuses on supervised learning using a trajectory based distance metric for category discrimination in an oscillatory neural network model. Classification is accomplished using a trajectory based distance metric. Since the distance metric is differentiable, a supervised learning algorithm based on gradient descent is demonstrated. Classification of spatiotemporal frequency transitions and their relation to a priori assessed categories is shown along with the improved classification results after supervised training. The results indicate that this spatiotemporal representation of stimuli and the associated distance metric is useful for simple pattern recognition tasks and that supervised learning improves classification results.     

An hierarchical artificial neural network system for the classification of transmembrane proteins.

This work presents a simple artificial neural network which classifies proteins into two classes from their sequences alone: the membrane protein class and the non-membrane protein class. This may be important in the functional assignment and analysis of open reading frames (ORF's) identified in complete genomes and, especially, those ORF's that correspond to proteins with unknown function. The network described here has a simple hierarchical feed-forward topology and a limited number of neurons which makes it very fast. By using only information contained in 11 protein sequences, the method was able to identify, with 100% accuracy, all membrane proteins with reliable topologies collected from several papers in the literature. Applied to a test set of 995 globular, water-soluble proteins, the neural network classified falsely 23 of them in the membrane protein class (97.7% of correct assignment). The method was also applied to the complete SWISS-PROT database with considerable success and on ORF's of several complete genomes. The neural network developed was associated with the PRED-TMR algorithm (Pasquier,C., Promponas,V.J., Palaios,G.A., Hamodrakas,J.S. and Hamodrakas,S.J., 1999) in a new application package called PRED-TMR2. A WWW server running the PRED-TMR2 software is available at http://o2.db.uoa.gr/PRED-TMR2     

On how network architecture determines the dominant patterns of spontaneous neural activity

In the absence of sensory stimulation, neocortical circuits display complex patterns of neural activity. These patterns are thought to reflect relevant properties of the network, including anatomical features like its modularity. It is also assumed that the synaptic connections of the network constrain the repertoire of emergent, spontaneous patterns. Although the link between network architecture and network activity has been extensively investigated in the last few years from different perspectives, our understanding of the relationship between the network connectivity and the structure of its spontaneous activity is still incomplete. Using a general mathematical model of neural dynamics we have studied the link between spontaneous activity and the underlying network architecture. In particular, here we show mathematically how the synaptic connections between neurons determine the repertoire of spatial patterns displayed in the spontaneous activity. To test our theoretical result, we have also used the model to simulate spontaneous activity of a neural network, whose architecture is inspired by the patchy organization of horizontal connections between cortical columns in the neocortex of primates and other mammals. The dominant spatial patterns of the spontaneous activity, calculated as its principal components, coincide remarkably well with those patterns predicted from the network connectivity using our theory. The equivalence between the concept of dominant pattern and the concept of attractor of the network dynamics is also demonstrated. This in turn suggests new ways of investigating encoding and storage capabilities of neural networks.