基于深度残差神经网络迁移学习的牙形刺图像识别

为了探索基于深度神经网络模型的牙形刺图像智能识别效果,研究选取奥陶纪8种牙形刺作为研究对象,通过体视显微镜采集牙形刺图像1188幅,收集整理公开发表文献的牙形刺图像778幅,将图像数据集划分为训练集和测试集。通过对训练集图像进行旋转、翻转、滤波增强处理,解决了训练样本不足的问题。基于ResNet-18、ResNet-34、ResNet-50、ResNet-101、ResNet-152五种残差神经网络模型,采用迁移学习方法,对网络模型进行训练以获取模型参数,五种模型测试Top-1准确率分别为85.37%、85.85%、83.90%、81.95%、80.00%, Top-2准确率分别为94.63%、94.63%、94.15%、93.17%、93.66%,模型对牙形刺图像具有较好的识别效果。通过对比研究发现,ResNet-34识别准确率最高,说明对于特征简单的牙形刺属种,增加网络深度并不一定能提升准确率,而确定合适深度的模型则不仅可以提高识别准确率,还可以节约计算资源。通过ResNet-34模型的迁移学习训练和重新训练效果对比可以看出,迁移学习不仅可以获得较高的准确率,而且可以较快获取模型参...     

Automating parallel implementation of neural learning algorithms

Neural learning algorithms generally involve a number of identical processing units, which are fully or partially connected, and involve an update function, such as a ramp, a sigmoid or a Gaussian function for instance. Some variations also exist, where units can be heterogeneous, or where an alternative update technique is employed, such as a pulse stream generator. Associated with connections are numerical values that must be adjusted using a learning rule, and and dictated by parameters that are learning rule specific, such as momentum, a learning rate, a temperature, amongst others. Usually, neural learning algorithms involve local updates, and a global interaction between units is often discouraged, except in instances where units are fully connected, or involve synchronous updates. In all of these instances, concurrency within a neural algorithm cannot be fully exploited without a suitable implementation strategy. A design scheme is described for translating a neural learning algorithm from inception to implementation on a parallel machine using PVM or MPI libraries, or onto programmable logic such as FPGAs. A designer must first describe the algorithm using a specialised Neural Language, from which a Petri net (PN) model is constructed automatically for verification, and building a performance model. The PN model can be used to study issues such as synchronisation points, resource sharing and concurrency within a learning rule. Specialised constructs are provided to enable a designer to express various aspects of a learning rule, such as the number and connectivity of neural nodes, the interconnection strategies, and information flows required by the learning algorithm. A scheduling and mapping strategy is then used to translate this PN model onto a multiprocessor template. We demonstrate our technique using a Kohonen and backpropagation learning rules, implemented on a loosely coupled workstation cluster, and a dedicated parallel machine, with PVM libraries.     

Identification of synthetic lethality based on a functional network by using machine learning algorithms

Synthetic lethality is the synthesis of mutations leading to cell death. Tumor-specific synthetic lethality has been targeted in research to improve cancer therapy. With the advances of techniques in molecular biology, such as RNAi and CRISPR/Cas9 gene editing, efforts have been made to systematically identify synthetic lethal interactions, especially for frequently mutated genes in cancers. However, elucidating the mechanism of synthetic lethality remains a challenge because of the complexity of its influencing conditions. In this study, we proposed a new computational method to identify critical functional features that can accurately predict synthetic lethal interactions. This method incorporates several machine learning algorithms and encodes protein-coding genes by an enrichment system derived from gene ontology terms and Kyoto Encyclopedia of Genes and Genomes pathways to represent their functional features. We built a random forest-based prediction engine by using 2120 selected features and obtained a Matthews correlation coefficient of 0.532. We examined the top 15 features and found that most of them have potential roles in synthetic lethality according to previous studies. These results demonstrate the ability of our proposed method to predict synthetic lethal interactions and provide a basis for further characterization of these particular genetic combinations.     

Recognition method of soybean leaf diseases using residual neural network based on transfer learning

Leaf disease is an important factor restricting the high quality and high yield of the soybean plant. Insufficient control of soybean diseases will destroy the local ecological environment and break the stability of the food chain. To overcome the low accuracy in recognizing soybean leaf diseases using traditional deep learning models and complexity in chemical analysis operations, in this study, a recognition model of soybean leaf diseases was proposed based on an improved deep learning model. First, four types of soybean diseases (Septoria Glycines Hemmi, Soybean Brown Leaf Spot, Soybean Frogeye Leaf Spot, and Soybean Phyllosticta Leaf Spot) were taken as research objects. Second, image preprocessing and data expansion of original images were carried out using image registration, image segmentation, region calibration and data enhancement. The data set containing 53, 250 samples was randomly divided into the training set, verification set, and test set according to the ratio of 7:2:1. Third, the convolution layer weight of the pre-training model based on the ImageNet open data set was transferred to the convolution layer of the ResNet18 model to reconstruct the global average pooling layer and the fully connected layer for constructing recognition model of TRNet18 model. Finally, the recognition accuracy of the four leaf diseases reached 99.53%, the Macro-F1 was 99.54%, and the average recognition time was 0.047184 s. Compared with AlexNet, ResNet18, ResNet50, and TRNet50 models, the recognition accuracy and Macro-F1 of the TRNet18 model were improved by 6.03% and 5.99% respectively, and the model recognition time was saved by 16.67%, The results showed that the proposed TRNet18 model had higher classification accuracy and stronger robustness, which can not only provide a reference for accurate recognition of other crop diseases, but also be transplanted to the mobile terminal for recognition of crop leaf diseases.     

Anti-Hebbian learning in a non-linear neural network

The Hebbian rule (Hebb 1949), coupled with an appropriate mechanism to limit the growth of synaptic weights, allows a neuron to learn to respond to the first principal component of the distribution of its input signals (Oja 1982). Rubner and Schulten (1990) have recently suggested the use of an anti-Hebbian rule in a network with hierarchical lateral connections. When applied to neurons with linear response functions, this model allows additional neurons to learn to respond to additional principal components (Rubner and Tavan 1989). Here we apply the model to neurons with non-linear response functions characterized by a threshold and a transition width. We propose local, unsupervised learning rules for the threshold and the transition width, and illustrate the operation of these rules with some simple examples. A network using these rules sorts the input patterns into classes, which it identifies by a binary code, with the coarser structure coded by the earlier neurons in the hierarchy.     

Oscillatory neural network for pattern recognition: trajectory based classification and supervised learning

Computer algorithms that match human performance in recognizing written text or spoken conversation remain elusive. The reasons why the human brain far exceeds any existing recognition scheme to date in the ability to generalize and to extract invariant characteristics relevant to category matching are not clear. However, it has been postulated that the dynamic distribution of brain activity (spatiotemporal activation patterns) is the mechanism by which stimuli are encoded and matched to categories. This research focuses on supervised learning using a trajectory based distance metric for category discrimination in an oscillatory neural network model. Classification is accomplished using a trajectory based distance metric. Since the distance metric is differentiable, a supervised learning algorithm based on gradient descent is demonstrated. Classification of spatiotemporal frequency transitions and their relation to a priori assessed categories is shown along with the improved classification results after supervised training. The results indicate that this spatiotemporal representation of stimuli and the associated distance metric is useful for simple pattern recognition tasks and that supervised learning improves classification results.     

Computational neural networks: Enhancing supervised learning algorithms via self-organization

A neural network processing scheme is proposed which utilizes a self-organizing Kohonen feature map as the front end to a feedforward classifier network. The results of a series of benchmarking studies based upon artificial statistical pattern recognition tasks indicate that the proposed architecture performs significantly better than conventional feedforward classifier networks when the decision regions are disjoint. This is attributed to the fact that the self-organization process allows internal units in the succeeding classifier network to be sensitive to a specific set of features in the input space at the outset of training.     

Abstractive morphological learning with a recurrent neural network

In traditional word-and-paradigm models of morphology, an inflectional system is represented via a set of exemplary paradigms. Novel wordforms are produced by analogy with previously encountered forms. This paper describes a recurrent neural network which can use this strategy to learn the paradigms of a morphologically complex language based on incomplete and randomized input. Results are given which show good performance for a range of typologically diverse languages.     

Predicting siRNA activity based on back-propagation neural network

RNA interference (RNAi) is a phenomenon of gene silence induced by a double-stranded RNA (dsRNA) homologous to a target gene. RNAi can be used to identify the function of genes or to knock down the targeted genes. In RNAi technology, 19 bp double-stranded short interfering RNAs (siRNA) with characteristic 39 overhangs are usually used. The effects of siRNAs are quite varied due to the different choices in the sites of target mRNA. Moreover, there are many factors influencing siRNA activity and these factors are usually nonlinear. To find the motif features and the effect on siRNA activity, we carried out a feature extraction on some published experimental data and used these features to train a back-propagation neural network (BP NN). Then, we used the trained BP NN to predict siRNA activity. __________ Translated from Acta Biophysica Sinica, 2006, 22(6): 429–434 [译自: 生物物理学报]     

基于SOFM网络的生态水文区划

生态水文区划对缓解区域水资源开发利用和生态环境保护之间矛盾起到了重要作用。本文基于自组织特征映射(self-organizing feature map,SOFM)人工神经网络建立了生态水文区划模型。首先运用主成分分析法从众多生态水文指标中提取出能代表绝大部分信息的主成分指标;其次依据提取的主成分指标,利用系统聚类得到区域聚类谱系图;而后构建SOFM神经网络,依据网络结果和系统聚类谱系图,划分合理的生态水文区。以福建省泉州市为例进行了生态水文区划研究,将研究区划分为4类区域,各区域具有明显的生态水文特征,针对不同区域特征,提出了常规、加强、较为严格和最为严格4种生态环境保护和水资源开发策略。     

Predicting siRNA activity based on back-propagation neural network

RNA interference (RNAi) is a phenomenon of gene silence induced by a double-stranded RNA (dsRNA) homologous to a target gene.RNAi can be used to identify the function of genes or to knock down the targeted genes.In RNAi technology,19 bp double-stranded short interfering RNAs (siRNA) with characteristic 3' overhangs are usually used.The effects of siRNAs are quite varied due to the different choices in the sites of target mRNA.Moreover,there are many factors influencing siRNA activity and these factors are usually nonlinear.To find the motif features and the effect on siRNA activity,we carried out a feature extraction on some published experimental data and used these features to train a backpropagation neural network (BP NN).Then,we used the trained BP NN to predict siRNA activity.     

Entropy learning and relevance criteria for neural network pruning

In this paper, entropy is a term used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence an entropy approach is proposed to dampen the early creation of such nodes by using a new computation called entropy cycle. Entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be pruned to reduce the memory requirements of the neural network.     

Template learning of cellular neural network using genetic programming

A new learning algorithm for space invariant Uncoupled Cellular Neural Network is introduced. Learning is formulated as an optimization problem. Genetic Programming has been selected for creating new knowledge because they allow the system to find new rules both near to good ones and far from them, looking for unknown good control actions. According to the lattice Cellular Neural Network architecture, Genetic Programming will be used in deriving the Cloning Template. Exploration of any stable domain is possible by the current approach. Details of the algorithm are discussed and several application results are shown.     

1988-2013年基于BP神经网络的植被叶面积指数遥感定量反演

叶面积指数（Leaf Area Index,LAI）高度综合了植被水平覆盖状况和垂直结构,以及枯枝落叶层厚薄和地下生物量多少,是植被影响土壤侵蚀的主要方面。区域尺度的时间序列叶面积指数揭示了区域土壤侵蚀的演化过程。因此,及时准确地掌握区域尺度上长时间序列的植被LAI,对研究土壤侵蚀动态变化与植被的关系至关重要。选择南京市1988-2013年10期遥感影像,基于反向传播（Back Propagation,BP）神经网络构建LAI反演模型,进行了长时间序列的叶面积指数反演。结合2009和2010年LAI实测值,验证与探讨了该模型的评价精度与适应性。结果表明：（1）该模型拟合度较高,2009和2010年平均相对误差、均方根误差、相关系数分别是0.2395和0.2174,0.2962和0.2581,0.7713和0.6844,各项精度评价指标均较好;（2）统计分析去除耕地后全市LAI变化,低植被覆盖（LAI<2）面积不断增加,高植被覆盖区（LAI>3）面积先减少后增加,耕地面积不断减少,符合南京市的发展变化规律;（3）主城区LAI年际变化与其他学者得到的南京市植被盖度变化趋势一致,反演结果的时序性较高。本文提出的基于反向传播神经网络模型反演长时间序列LAI是可行的,为区域尺度土壤侵蚀定量遥感监测提供新途径。     

Comparison of different neural network algorithms in the diagnosis of acute appendicitis

Four different neural network algorithms, binary adaptive resonance theory (ART1), self-organizing map, learning vector quantization and back-propagation, were compared in the diagnosis of acute appendicitis with different parameter groups. The results show that supervised learning algorithms learning vector quantization and back-propagation were better than unsupervised algorithms in this medical decision making problem. The best results were obtained with the learning vector quantization. The self-organizing map algorithm showed good specificity, but this was in conjunction with lower sensitivity. The best parameter group was found to be the clinical signs. It seems beneficial to design a decision support system which uses these methods in the decision making process.     

Artificial neural network learning of nonstationary behavior in time series

We refine and complement a previously-proposed artificial neural network method for learning hidden signals forcing nonstationary behavior in time series. The method adds an extra input unit to the network and feeds it with the proposed profile for the unknown perturbing signal. The correct time evolution of this new input parameter is learned simultaneously with the intrinsic stationary dynamics underlying the series, which is accomplished by minimizing a suitably-defined error function for the training process. We incorporate here the use of validation data, held out from the training set, to accurately determine the optimal value of a hyperparameter required by the method. Furthermore, we evaluate this algorithm in a controlled situation and show that it outperforms other existing methods in the literature. Finally, we discuss a preliminary application to the real-world sunspot time series and link the obtained hidden perturbing signal to the secular evolution of the solar magnetic field.