A neural network architecture for data classification

This article aims at showing an architecture of neural networks designed for the classification of data distributed among a high number of classes. A significant gain in the global classification rate can be obtained by using our architecture. This latter is based on a set of several little neural networks, each one discriminating only two classes. The specialization of each neural network simplifies their structure and improves the classification. Moreover, the learning step automatically determines the number of hidden neurons. The discussion is illustrated by tests on databases from the UCI machine learning database repository. The experimental results show that this architecture can achieve a faster learning, simpler neural networks and an improved performance in classification.     

A neural network model with feature selection for Korean speech act classification

A speech act is a linguistic action intended by a speaker. Speech act classification is an essential part of a dialogue understanding system because the speech act of an utterance is closely tied with the user's intention in the utterance. We propose a neural network model for Korean speech act classification. In addition, we propose a method that extracts morphological features from surface utterances and selects effective ones among the morphological features. Using the feature selection method, the proposed neural network can partially increase precision and decrease training time. In the experiment, the proposed neural network showed better results than other models using comparatively high-level linguistic features. Based on the experimental result, we believe that the proposed neural network model is suitable for real field applications because it is easy to expand the neural network model into other domains. Moreover, we found that neural networks can be useful in speech act classification if we can convert surface sentences into vectors with fixed dimensions by using an effective feature selection method.     

Oscillatory neural network for pattern recognition: trajectory based classification and supervised learning

Computer algorithms that match human performance in recognizing written text or spoken conversation remain elusive. The reasons why the human brain far exceeds any existing recognition scheme to date in the ability to generalize and to extract invariant characteristics relevant to category matching are not clear. However, it has been postulated that the dynamic distribution of brain activity (spatiotemporal activation patterns) is the mechanism by which stimuli are encoded and matched to categories. This research focuses on supervised learning using a trajectory based distance metric for category discrimination in an oscillatory neural network model. Classification is accomplished using a trajectory based distance metric. Since the distance metric is differentiable, a supervised learning algorithm based on gradient descent is demonstrated. Classification of spatiotemporal frequency transitions and their relation to a priori assessed categories is shown along with the improved classification results after supervised training. The results indicate that this spatiotemporal representation of stimuli and the associated distance metric is useful for simple pattern recognition tasks and that supervised learning improves classification results.     

A bayesian recurrent neural network for unsupervised pattern recognition in large incomplete data sets

A recurrent neural network, modified to handle highly incomplete training data is described. Unsupervised pattern recognition is demonstrated in the WHO database of adverse drug reactions. Comparison is made to a well established method, AutoClass, and the performances of both methods is investigated on simulated data. The neural network method performs comparably to AutoClass in simulated data, and better than AutoClass in real world data. With its better scaling properties, the neural network is a promising tool for unsupervised pattern recognition in huge databases of incomplete observations.     

A novel method based on neural architecture search for Diptera insect classification on embedded devices

Diptera insects have the characteristics of spreading diseases and destroying forests. There are similarities among different species, which makes it difficult to identify a Diptera insect. Most traditional convolutional neural networks have large parameters and high recognition latency. Therefore, they are not suitable for deploying models on embedded devices for classification and recognition. This paper proposes an improved neural architecture based on differentiable search method. First, we designed a network search cell by adding the feature output of the previous layer to each search cell. Second, we added the attention module to the search space to expand the searchable range. At the same time, we used methods such as model quantization and limiting the ReLU function to the ReLU6 function to reduce computer resource consumption. Finally, the network model was transplanted to the NVIDIA Jetson Xavier NX embedded development platform to verify the network performance so that the neural architecture search could be organically combined with the embedded development platform. The experimental results show that the designed neural architecture achieves 98.9% accuracy on the Diptera insect dataset with a latency of 8.4 ms. It has important practical significance for the recognition of Diptera insects in embedded devices.     

A combinational feature selection and ensemble neural network method for classification of gene expression data

Background  

Microarray experiments are becoming a powerful tool for clinical diagnosis, as they have the potential to discover gene expression patterns that are characteristic for a particular disease. To date, this problem has received most attention in the context of cancer research, especially in tumor classification. Various feature selection methods and classifier design strategies also have been generally used and compared. However, most published articles on tumor classification have applied a certain technique to a certain dataset, and recently several researchers compared these techniques based on several public datasets. But, it has been verified that differently selected features reflect different aspects of the dataset and some selected features can obtain better solutions on some certain problems. At the same time, faced with a large amount of microarray data with little knowledge, it is difficult to find the intrinsic characteristics using traditional methods. In this paper, we attempt to introduce a combinational feature selection method in conjunction with ensemble neural networks to generally improve the accuracy and robustness of sample classification.     

A neural network based pattern recognition system for somatic embryos of Douglas fir

A pattern recognition system was developed to classify Douglas fir somatic embryos by employing an image analysis system and two neural network based classifiers. The contour of embryo images was segmented, digitalized and converted to numerical values after the discrete and fast Fourier transformation. These values, or Fourier features, along with some other shape factors, were used for embryo classification. The pattern recognition system used a hierarchical decision tree to classify Douglas fir embryos into three normal and one abnormal embryo classes. An accuracy of greater than 80% was achieved for normal embryos. This system provides an objective and efficient method of classifying embryos of Douglas fir. It will be a useful tool for kinetic studies and process optimization of conifer somatic embryogenesis.     

FuzzyART neural network for protein classification

One of the major research directions in bioinformatics is that of predicting the protein superfamily in large databases and classifying a given set of protein domains into superfamilies. The classification reflects the structural, evolutionary and functional relatedness. These relationships are embodied in hierarchical classification such as Structural Classification of Protein (SCOP), which is manually curated. Such classification is essential for the structural and functional analysis of proteins. Yet, a large number of proteins remain unclassified. We have proposed an unsupervised machine-learning FuzzyART neural network algorithm to classify a given set of proteins into SCOP superfamilies. The proposed method is fast learning and uses an atypical non-linear pattern recognition technique. In this approach, we have constructed a similarity matrix from p-values of BLAST all-against-all, trained the network with FuzzyART unsupervised learning algorithm using the similarity matrix as input vectors and finally the trained network offers SCOP superfamily level classification. In this experiment, we have evaluated the performance of our method with existing techniques on six different datasets. We have shown that the trained network is able to classify a given similarity matrix of a set of sequences into SCOP superfamilies at high classification accuracy.     

A model of neural network for spatiotemporal pattern recognition

A model of neural network to recognize spatiotemporal patterns is presented. The network consists of two kinds of neural cells: P-cells and B-cells. A P-cell generates an impulse responding to more than one impulse and embodies two special functions: short term storage (STS) and heterosynaptic facilitation (HSF). A B-cell generates several impulses with high frequency as soon as it receives an impulse. In recognizing process, an impulse generated by a P-cell represents a recognition of stimulus pattern, and triggers the generation of impulses of a B-cell. Inhibitory impulses with high frequency generated by a B-cell reset the activities of all P-cells in the network.Two examples of spatiotemporal pattern recognition are presented. They are achieved by giving different values to the parameters of the network. In one example, the network recognizes both directional and non-directional patterns. The selectivities to directional and non-directional patterns are realized by only adjusting excitatory synaptic weights of P-cells. In the other example, the network recognizes time series of spatial patterns, where the lengths of the series are not necessarily the same and the transitional speeds of spatial patterns are not always the same. In both examples, the HSF signal controls the total activity of the network, which contributes to exact recognition and error recovery. In the latter example, it plays a role to trigger and execute the recognizing process. Finally, we discuss the correspondence between the model and physiological findings.     

Semantic region of interest and species classification in the deep neural network feature domain

In this paper, we focus on animal object detection and species classification in camera-trap images collected in highly cluttered natural scenes. Using a deep neural network (DNN) model training for animal- background image classification, we analyze the input camera-trap images to generate a multi-level visual representation of the input image. We detect semantic regions of interest for animals from this representation using k-mean clustering and graph cut in the DNN feature domain. These animal regions are then classified into animal species using multi-class deep neural network model. According the experimental results, our method achieves 99.75% accuracy for classifying animals and background and 90.89% accuracy for classifying 26 animal species on the Snapshot Serengeti dataset, outperforming existing image classification methods.     

基于分类知识利用神经网络反演叶面积指数

叶面积指数（LAI,Leaf Area Index）是陆面过程中一个十分重要的输入参数,其遥感反演方法研究一直是国内外遥感应用研究的热点问题。基于统计的遥感反演方法由于缺乏物理基础,其可靠性和普适性差。基于物理的冠层反射模型的LAI反演方法克服了上述弊端,但是由于反演过程是病态的,模型反演结果一般不唯一。神经网络算法的介入可在一定程度上改善这一问题,但是模型反演的病态问题至今仍无法很好地解决。在PROSAIL模型敏感性分析的基础上提出了一种基于影像分类的神经网络反演方法,引进了土壤反射指数用于替代原模型中难以确定的土壤背景反射参数,分别针对不同植被类型建立各自的神经网络,对经过大气纠正后的Landsat ETM＋影像进行了模拟实验并同野外实测LAI数据进行比较。结果表明,对于LAI小于3的植被区该方法的反演精度比较可靠,而LAI大于3的植被区,反演的LAI偏小,原因归结为密植被的冠层反射在LAI大于3以后趋于饱和而无法敏感地表征LAI的变化所导致的。     

A neural network model for selective attention in visual pattern recognition

A neural network model of the mechanism of selective attention in visual pattern recognition is proposed and simulated on a digital computer.When a complex figure consisting of two patterns or more is presented to the model, it is segmented into individual patterns, and each pattern is recognized separately. Even if one of the patterns to which the model is paying selective attention is affected by noise or defects, the model can recall the complete pattern from which the noise has been eliminated and the defects corrected. It is not necessary for perfect recall that the stimulus pattern should be identical in shape to the training pattern. Even though the pattern is distorted in shape or changed in size, it can be correctly recognized and the missing portions restored.The model consists of a hierarchical neural network which has efferent as well as afferent connections between cells. The afferent and the efferent signals interact with each other in the network: the efferent signals, that is, the signals for selective attention, have a facilitating effect on the afferent ones, and, at the same time, the afferent signals gate efferent signal flow. When some feature in the stimulus is not extracted in the afferent paths, the threshold for detection of that feature is automatically lowered by decreasing the efficiency of inhibition, and the model tries to extract even vague traces of the undetected feature.     

Novel neural network application for bacterial colony classification

Background

Bacterial colony morphology is the first step of classifying the bacterial species before sending them to subsequent identification process with devices, such as VITEK 2 automated system and mass spectrometry microbial identification system. It is essential as a pre-screening process because it can greatly reduce the scope of possible bacterial species and will make the subsequent identification more specific and increase work efficiency in clinical bacteriology. But this work needs adequate clinical laboratory expertise of bacterial colony morphology, which is especially difficult for beginners to handle properly. This study presents automatic programs for bacterial colony classification task, by applying the deep convolutional neural networks (CNN), which has a widespread use of digital imaging data analysis in hospitals. The most common 18 bacterial colony classes from Peking University First Hospital were used to train this framework, and other images out of these training dataset were utilized to test the performance of this classifier.

Results

The feasibility of this framework was verified by the comparison between predicted result and standard bacterial category. The classification accuracy of all 18 bacteria can reach 73%, and the accuracy and specificity of each kind of bacteria can reach as high as 90%.

Conclusions

The supervised neural networks we use can have more promising classification characteristics for bacterial colony pre-screening process, and the unsupervised network should have more advantages in revealing novel characteristics from pictures, which can provide some practical indications to our clinical staffs.

     

Bacterial classification with convolutional neural networks based on different data reduction layers

Abstract

For high accuracy classification of DNA sequences through Convolutional Neural Networks (CNNs), it is essential to use an efficient sequence representation that can accelerate similarity comparison between DNA sequences. In addition, CNN networks can be improved by avoiding the dimensionality problem associated with multi-layer CNN features. This paper presents a new approach for classification of bacterial DNA sequences based on a custom layer. A CNN is used with Frequency Chaos Game Representation (FCGR) of DNA. The FCGR is adopted as a sequence representation method with a suitable choice of the frequency k-lengthen words occurrence in DNA sequences. The DNA sequence is mapped using FCGR that produces an image of a gene sequence. This sequence displays both local and global patterns. A pre-trained CNN is built for image classification. First, the image is converted to feature maps through convolutional layers. This is sometimes followed by a down-sampling operation that reduces the spatial size of the feature map and removes redundant spatial information using the pooling layers. The Random Projection (RP) with an activation function, which carries data with a decent variety with some randomness, is suggested instead of the pooling layers. The feature reduction is achieved while keeping the high accuracy for classifying bacteria into taxonomic levels. The simulation results show that the proposed CNN based on RP has a trade-off between accuracy score and processing time.     

JACOP: A simple and robust method for the automated classification of protein sequences with modular architecture

Background  

Whole-genome sequencing projects are rapidly producing an enormous number of new sequences. Consequently almost every family of proteins now contains hundreds of members. It has thus become necessary to develop tools, which classify protein sequences automatically and also quickly and reliably. The difficulty of this task is intimately linked to the mechanism by which protein sequences diverge, i.e. by simultaneous residue substitutions, insertions and/or deletions and whole domain reorganisations (duplications/swapping/fusion).     

A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression

This paper studies the problem of building multiclass classifiers for tissue classification based on gene expression. The recent development of microarray technologies has enabled biologists to quantify gene expression of tens of thousands of genes in a single experiment. Biologists have begun collecting gene expression for a large number of samples. One of the urgent issues in the use of microarray data is to develop methods for characterizing samples based on their gene expression. The most basic step in the research direction is binary sample classification, which has been studied extensively over the past few years. This paper investigates the next step-multiclass classification of samples based on gene expression. The characteristics of expression data (e.g. large number of genes with small sample size) makes the classification problem more challenging. The process of building multiclass classifiers is divided into two components: (i) selection of the features (i.e. genes) to be used for training and testing and (ii) selection of the classification method. This paper compares various feature selection methods as well as various state-of-the-art classification methods on various multiclass gene expression datasets. Our study indicates that multiclass classification problem is much more difficult than the binary one for the gene expression datasets. The difficulty lies in the fact that the data are of high dimensionality and that the sample size is small. The classification accuracy appears to degrade very rapidly as the number of classes increases. In particular, the accuracy was very low regardless of the choices of the methods for large-class datasets (e.g. NCI60 and GCM). While increasing the number of samples is a plausible solution to the problem of accuracy degradation, it is important to develop algorithms that are able to analyze effectively multiple-class expression data for these special datasets.     

Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors

This paper analyses a model for the parallel development and adult coding of neural feature detectors. The model was introduced in Grossberg (1976). We show how experience can retune feature detectors to respond to a prescribed convex set of spatial patterns. In particular, the detectors automatically respond to average features chosen from the set even if the average features have never been experienced. Using this procedure, any set of arbitrary spatial patterns can be recoded, or transformed, into any other spatial patterns (universal recoding), if there are sufficiently many cells in the network's cortex. The network is built from short term memory (STM) and long term memory (LTM) mechanisms, including mechanisms of adaptation, filtering, contrast enhancement, tuning, and nonspecific arousal. These mechanisms capture some experimental properties of plasticity in the kitten visual cortex. The model also suggests a classification of adult feature detector properties in terms of a small number of functional principles. In particular, experiments on retinal dynamics, including amarcrine cell function, are suggested.Supported in part by the Advanced Research Projects Agency under ONR Contract No. N00014-76-C-0185     

Automated classification of urinary stones based on microcomputed tomography images using convolutional neural network

PurposeThe classification of urinary stones is important prior to treatment because the treatments depend on three types of urinary stones, i.e., calcium, uric acid, and mixture stones. We have developed an automatic approach for the classification of urinary stones into the three types based on microcomputed tomography (micro-CT) images using a convolutional neural network (CNN).Materials and methodsThirty urinary stones from different patients were scanned in vitro using micro-CT (pixel size: 14.96 μm; slice thickness: 15 μm); a total of 2,430 images (micro-CT slices) were produced. The slices (227 × 227 pixels) were classified into the three categories based on their energy dispersive X-ray (EDX) spectra obtained via scanning electron microscopy (SEM). The images of urinary stones from each category were divided into three parts; 66%, 17%, and 17% of the dataset were assigned to the training, validation, and test datasets, respectively. The CNN model with 15 layers was assessed based on validation accuracy for the optimization of hyperparameters such as batch size, learning rate, and number of epochs with different optimizers. Then, the model with the optimized hyperparameters was evaluated for the test dataset to obtain classification accuracy and error.ResultsThe validation accuracy of the developed approach with CNN with optimized hyperparameters was 0.9852. The trained CNN model achieved a test accuracy of 0.9959 with a classification error of 1.2%.ConclusionsThe proposed automated CNN-based approach could successfully classify urinary stones into three types, namely calcium, uric acid, and mixture stones, using micro-CT images.