An improvement of extreme learning machine for compact single-hidden-layer feedforward neural networks

Recently, a novel learning algorithm called extreme learning machine (ELM) was proposed for efficiently training single-hidden-layer feedforward neural networks (SLFNs). It was much faster than the traditional gradient-descent-based learning algorithms due to the analytical determination of output weights with the random choice of input weights and hidden layer biases. However, this algorithm often requires a large number of hidden units and thus slowly responds to new observations. Evolutionary extreme learning machine (E-ELM) was proposed to overcome this problem; it used the differential evolution algorithm to select the input weights and hidden layer biases. However, this algorithm required much time for searching optimal parameters with iterative processes and was not suitable for data sets with a large number of input features. In this paper, a new approach for training SLFNs is proposed, in which the input weights and biases of hidden units are determined based on a fast regularized least-squares scheme. Experimental results for many real applications with both small and large number of input features show that our proposed approach can achieve good generalization performance with much more compact networks and extremely high speed for both learning and testing.     

Gaussian processes for machine learning

Gaussian processes (GPs) are natural generalisations of multivariate Gaussian random variables to infinite (countably or continuous) index sets. GPs have been applied in a large number of fields to a diverse range of ends, and very many deep theoretical analyses of various properties are available. This paper gives an introduction to Gaussian processes on a fairly elementary level with special emphasis on characteristics relevant in machine learning. It draws explicit connections to branches such as spline smoothing models and support vector machines in which similar ideas have been investigated. Gaussian process models are routinely used to solve hard machine learning problems. They are attractive because of their flexible non-parametric nature and computational simplicity. Treated within a Bayesian framework, very powerful statistical methods can be implemented which offer valid estimates of uncertainties in our predictions and generic model selection procedures cast as nonlinear optimization problems. Their main drawback of heavy computational scaling has recently been alleviated by the introduction of generic sparse approximations.13,78,31 The mathematical literature on GPs is large and often uses deep concepts which are not required to fully understand most machine learning applications. In this tutorial paper, we aim to present characteristics of GPs relevant to machine learning and to show up precise connections to other "kernel machines" popular in the community. Our focus is on a simple presentation, but references to more detailed sources are provided.     

Development of a sensitivity analysis framework for aquatic biogeochemical models using machine learning

Our evolving understanding of ecosystem functioning along with the advent of computational power have paved the way for the development of complex mathematical models that explicitly represent the functional diversity of biotic communities and multiple biogeochemical cycles. The ever-growing demand for more complex models underscores the importance of robust sensitivity analysis (SA) to elucidate the impact of the uncertainty on model inputs and to untangle the parameter covariance patterns that ultimately lead to the emergence of equifinality problems. In this study, we propose a novel multi-pronged SA framework that integrates advanced statistical and machine learning (ML) techniques. Principal component analysis (PCA) is first applied to dissect the wide array of predictive outputs and identify modes of variability in time and/or space. Classification and Regression Tree (CART) analysis is then used to identify a set of splitting decisions connecting threshold values of key state variables and model parameters with different ranges of predictive outputs with management interest. Self-Organizing Maps (SOM) are implemented as a final step to unravel any non-linear associations between model parameters and responses. As a proof-of-concept, we used a complex aquatic biogeochemical model developed for the Bay of Quinte, a eutrophic embayment in Lake Ontario, to examine competition patterns and structural shifts among multiple functional phytoplankton (diatoms, N-fixing cyanobacteria, and Microcystis) and zooplankton (herbivores and omnivores) groups. Our sensitivity analysis framework showed that the parameters representing the dependence of growth and metabolic processes on temperature are particularly influential to recreate plankton community dynamics during the cold period of the year, whereas the interplay among the interspecific resource competition, strength of the prey-predator interactions, and phosphorus availability mainly regulate their dynamics during the growing season. The growth strategies of diatoms, their nutritional quality that determines the assimilation efficiency by zooplankton, along with the ambient nutrient availability determine our capacity to reproduce patterns of cyanobacteria dominance and faithfully depict the severity of harmful algal blooms. Finally, our study discusses the benefits of a broader use of the ML-based SA framework to unravel influential parametric interactions in modulating the behaviors of complex mathematical models.     

Adaptive machine learning for protein engineering

Machine-learning models that learn from data to predict how protein sequence encodes function are emerging as a useful protein engineering tool. However, when using these models to suggest new protein designs, one must deal with the vast combinatorial complexity of protein sequences. Here, we review how to use a sequence-to-function machine-learning surrogate model to select sequences for experimental measurement. First, we discuss how to select sequences through a single round of machine-learning optimization. Then, we discuss sequential optimization, where the goal is to discover optimized sequences and improve the model across multiple rounds of training, optimization, and experimental measurement.     

Beetles, boxes and brain cells: neural mechanisms underlying valuation and learning

Sensory cues in the environment can predict the availability of reward. Through experience, humans and animals learn these predictions and use them to guide their actions. For example, we can learn to discriminate chanterelles from ordinary champignons through experience. Assuming the development of a taste for the complex and lingering flavors of chanterelles, we therefore learn to value the same action--picking mushrooms--differentially depending upon the appearance of a mushroom. One major goal of cognitive neuroscience is to understand the neural mechanisms that underlie this sort of learning. Because the acquisition of rewards motivates much behavior, recent efforts have focused on describing the neural signals related to learning the value of stimuli and actions. Neurons in the basal ganglia, in midbrain dopamine areas, in frontal and parietal cortices and in other brain areas, all modulate their activity in relation to aspects of learning. By training monkeys on various behavioral tasks, recent studies have begun to characterize how neural signals represent distinct processes, such as the timing of events, motivation, absolute (objective) and relative (subjective) valuation, and the formation of associative links between stimuli and potential actions. In addition, a number of studies have either further characterized dopamine signals or sought to determine how such signaling might interact with target structures, such as the striatum and rhinal cortex, to underlie learning.     

A neural model for category learning

We present a general neural model for supervised learning of pattern categories which can resolve pattern classes separated by nonlinear, essentially arbitrary boundaries. The concept of a pattern class develops from storing in memory a limited number of class elements (prototypes). Associated with each prototype is a modifiable scalar weighting factor () which effectively defines the threshold for categorization of an input with the class of the given prototype. Learning involves (1) commitment of prototypes to memory and (2) adjustment of the various factors to eliminate classification errors. In tests, the model ably defined classification boundaries that largely separated complicated pattern regions. We discuss the role which divisive inhibition might play in a possible implementation of the model by a network of neurons.This work was supported in part by the Alfred P. Sloan Foundation and the Ittleson Foundation, Inc.     

The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning

Dopamine is thought to play a major role in learning. However, while dopamine D1 receptors (D1Rs) in the prefrontal cortex (PFC) have been shown to modulate working memory-related neural activity, their role in the cellular basis of learning is unknown. We recorded activity from multiple electrodes while injecting the D1R antagonist SCH23390 in the lateral PFC as monkeys learned visuomotor associations. Blocking D1Rs impaired learning of novel associations and decreased cognitive flexibility but spared performance of already familiar associations. This suggests a greater role for prefrontal D1Rs in learning new, rather than performing familiar, associations. There was a corresponding greater decrease in neural selectivity and increase in alpha and beta oscillations in local field potentials for novel than for familiar associations. Our results suggest that weak stimulation of D1Rs observed in aging and psychiatric disorders may impair learning and PFC function by reducing neural selectivity and exacerbating neural oscillations associated with inattention and cognitive deficits.     

Schizophrenia: neural mechanisms for novel therapies

Although valuable antischizophrenic drugs exist, they only partially ameliorate symptoms and elicit substantial side effects. Classic neuroleptic drugs act by blocking dopamine receptors. They can relieve some symptoms but not behavioral withdrawal features that are designated "negative" symptoms. Clozapine and related newer atypical neuroleptics may be more efficacious in relieving negative symptoms. Understandng their actions may facilitate new drug discovery. Agents influencing glutamate neurotransmission and N-methyl-D-aspartate receptors, especially the cotransmitter D-serine, are promising. Stimulation of the alpha7 subtype of nicotinic acetylcholine receptor may also be efficacious. The search for genes linked to schizophrenia has revealed several leads that may permit development of novel therapeutic agents. Promising genes include disrupted-in-schizophrenia-1, dysbindin, and neuregulin.     

Towards machine learning for hydrogel drug delivery systems

Hydrogel drug delivery system development is complex and laborious, and machine learning (ML) techniques hold great promise in accelerating the process. We highlight recent advances and strategies for data collection and ML, and we discuss the potential for and barriers to the broader use of ML for hydrogel drug delivery systems.     

A neural model for conditioned avoidance learning

A mathematical model for learning of a conditioned avoidance behavior is presented. An identification of the net excitation of a neural model (Rashevsky, N., 1960.Mathematical Biophysics. Vol. II. New York: Dover Publications, Inc.) with the instantaneous probability of response is introduced and its usefulness in discussing block-trial learning performances in the conditioned avoidance situation is outlined for normal and brain-operated animals, using experimental data collected by the author. Later, the model is applied to consecutive trial learning and connection is made with the approach of H. D. Landahl (1964. “An Avoidance Learning Situation. A Neural Net Model.”Bull. Math. Biophysics,26, 83–89; and 1965, “A Neural Net Model for Escape Learning.”Bull. Math. Biophysics,27, Special Edition, 317–328) wherein lie further data with which the model can be compared.     

机器学习在蛋白质功能预测领域的研究进展

蛋白质是有机生命体内不可或缺的化合物,在生命活动中发挥着多种重要作用,了解蛋白质的功能有助于医学和药物研发等领域的研究。此外,酶在绿色合成中的应用一直备受人们关注,但是由于酶的种类和功能多种多样,获取特定功能酶的成本高昂,限制了其进一步的应用。目前,蛋白质的具体功能主要通过实验表征确定,该方法实验工作繁琐且耗时耗力,同时,随着生物信息学和测序技术的高速发展,已测序得到的蛋白质序列数量远大于功能获得注释的序列数量,高效预测蛋白质功能变得至关重要。随着计算机技术的蓬勃发展,由数据驱动的机器学习方法已成为应对这些挑战的有效解决方案。本文对蛋白质功能及其注释方法以及机器学习的发展历程和操作流程进行了概述,聚焦于机器学习在酶功能预测领域的应用,对未来人工智能辅助蛋白质功能高效研究的发展方向提出了展望。     

Kernel-based machine learning protocol for predicting DNA-binding proteins

DNA-binding proteins (DNA-BPs) play a pivotal role in various intra- and extra-cellular activities ranging from DNA replication to gene expression control. Attempts have been made to identify DNA-BPs based on their sequence and structural information with moderate accuracy. Here we develop a machine learning protocol for the prediction of DNA-BPs where the classifier is Support Vector Machines (SVMs). Information used for classification is derived from characteristics that include surface and overall composition, overall charge and positive potential patches on the protein surface. In total 121 DNA-BPs and 238 non-binding proteins are used to build and evaluate the protocol. In self-consistency, accuracy value of 100% has been achieved. For cross-validation (CV) optimization over entire dataset, we report an accuracy of 90%. Using leave 1-pair holdout evaluation, the accuracy of 86.3% has been achieved. When we restrict the dataset to less than 20% sequence identity amongst the proteins, the holdout accuracy is achieved at 85.8%. Furthermore, seven DNA-BPs with unbounded structures are all correctly predicted. The current performances are better than results published previously. The higher accuracy value achieved here originates from two factors: the ability of the SVM to handle features that demonstrate a wide range of discriminatory power and, a different definition of the positive patch. Since our protocol does not lean on sequence or structural homology, it can be used to identify or predict proteins with DNA-binding function(s) regardless of their homology to the known ones.     

A neural net model for escape learning

An escape learning situation is discussed in terms of a neural model in which a stimulus can result in a conditioned excitement and a specific conditioned response. By using the simplest relations between the strengths of conditioning and the number of reinforcements and by introducing a distribution of fluctuations occurring regularly in time, one can calculate the probabilities of various responses, as well as the various latencies, in successive trials. The results are in moderately satisfactory agreement with the data of R. L. Solomon and L. C. Wynne (Psychol. Monogr.,67, No. 4, 1953). Consequences of the model for various experimental situations are discussed. This research was supported in part by the United States Public Health Service Grant RCA GM K6 18,420 and in part by the United States Air Force through the Air Force Office of Scientific Research of the Air Research Development Command under Grant No. AF AFOSR 370-64.     

环境微生物研究中机器学习算法及应用

微生物在环境中无处不在，它们不仅是生物地球化学循环和环境演化的关键参与者，也在环境监测、生态治理和保护中发挥着重要作用。随着高通量技术的发展，大量微生物数据产生，运用机器学习对环境微生物大数据进行建模和分析，在微生物标志物识别、污染物预测和环境质量预测等领域的科学研究和社会应用方面均具有重要意义。机器学习可分为监督学习和无监督学习2大类。在微生物组学研究当中，无监督学习通过聚类、降维等方法高效地学习输入数据的特征，进而对微生物数据进行整合和归类。监督学习运用有特征和标记的微生物数据集训练模型，在面对只有特征没有标记的数据时可以判断出标记，从而实现对新数据的分类、识别和预测。然而，复杂的机器学习算法通常以牺牲可解释性为代价来重点关注模型预测的准确性。机器学习模型通常可以看作预测特定结果的“黑匣子”，即对模型如何得出预测所知甚少。为了将机器学习更多地运用于微生物组学研究、提高我们提取有价值的微生物信息的能力，深入了解机器学习算法、提高模型的可解释性尤为重要。本文主要介绍在环境微生物领域常用的机器学习算法和基于微生物组数据的机器学习模型的构建步骤，包括特征选择、算法选择、模型构建和评估等，并对各种机器学习模型在环境微生物领域的应用进行综述，深入探究微生物组与周围环境之间的关联，探讨提高模型可解释性的方法，并为未来环境监测、环境健康预测提供科学参考。     

Recruiting machine learning methods for molecular simulations of proteins

Abstract

Molecular dynamics (MD) simulations are critical to understanding the movements of proteins in time. Yet, MD simulations are limited due to the availability of high-resolution protein structures, accuracy of the underlying force-field, computational expense, and difficulty in analysing big data-sets. Machine learning algorithms are now routinely used to circumvent many of these limitations and computational biophysicists are continuously making progress in developing novel applications. Here, we discuss some of these methods, varying from traditional dimensionality reduction approaches to more recent abstractions such as transfer learning and reinforcement learning, and how they have been used to deal with the challenges in MD. We conclude with the prospective issues in the application of machine learning methods in MD, to increase accuracy and efficiency of protein dynamics studies in general.     

Interpretable machine learning for high-dimensional trajectories of aging health

We have built a computational model for individual aging trajectories of health and survival, which contains physical, functional, and biological variables, and is conditioned on demographic, lifestyle, and medical background information. We combine techniques of modern machine learning with an interpretable interaction network, where health variables are coupled by explicit pair-wise interactions within a stochastic dynamical system. Our dynamic joint interpretable network (DJIN) model is scalable to large longitudinal data sets, is predictive of individual high-dimensional health trajectories and survival from baseline health states, and infers an interpretable network of directed interactions between the health variables. The network identifies plausible physiological connections between health variables as well as clusters of strongly connected health variables. We use English Longitudinal Study of Aging (ELSA) data to train our model and show that it performs better than multiple dedicated linear models for health outcomes and survival. We compare our model with flexible lower-dimensional latent-space models to explore the dimensionality required to accurately model aging health outcomes. Our DJIN model can be used to generate synthetic individuals that age realistically, to impute missing data, and to simulate future aging outcomes given arbitrary initial health states.