Mutually beneficial confluence of structure-based modeling of protein dynamics and machine learning methods

Proteins sample an ensemble of conformers under physiological conditions, having access to a spectrum of modes of motions, also called intrinsic dynamics. These motions ensure the adaptation to various interactions in the cell, and largely assist in, if not determine, viable mechanisms of biological function. In recent years, machine learning frameworks have proven uniquely useful in structural biology, and recent studies further provide evidence to the utility and/or necessity of considering intrinsic dynamics for increasing their predictive ability. Efficient quantification of dynamics-based attributes by recently developed physics-based theories and models such as elastic network models provides a unique opportunity to generate data on dynamics for training ML models towards inferring mechanisms of protein function, assessing pathogenicity, or estimating binding affinities.     

Nonlinear machine learning in simulations of soft and biological materials

Abstract

Interpretable parameterisations of free energy landscapes for soft and biological materials calculated from molecular simulation require the availability of ‘good’ collective variables (CVs) capable of discriminating the metastable states of the system and the barriers between them. If these CVs are coincident with the slow collective modes governing the long-time dynamical evolution, then they also furnish good coordinates in which to perform enhanced sampling to surmount high free energy barriers and efficiently explore and recover the landscape. Non-linear manifold learning techniques provide a means to systematically extract such CVs from molecular simulation trajectories by identifying and extracting low-dimensional manifolds lying latent within the high-dimensional coordinate space. We survey recent advances in data-driven CV discovery and enhanced sampling using non-linear manifold learning, describe the mathematical and theoretical underpinnings of these techniques, and present illustrative examples to molecular folding and colloidal self-assembly. We close with our outlook and perspective on future advances in this rapidly evolving field.     

Recruiting machine learning methods for molecular simulations of proteins

Abstract

Molecular dynamics (MD) simulations are critical to understanding the movements of proteins in time. Yet, MD simulations are limited due to the availability of high-resolution protein structures, accuracy of the underlying force-field, computational expense, and difficulty in analysing big data-sets. Machine learning algorithms are now routinely used to circumvent many of these limitations and computational biophysicists are continuously making progress in developing novel applications. Here, we discuss some of these methods, varying from traditional dimensionality reduction approaches to more recent abstractions such as transfer learning and reinforcement learning, and how they have been used to deal with the challenges in MD. We conclude with the prospective issues in the application of machine learning methods in MD, to increase accuracy and efficiency of protein dynamics studies in general.     

Recent developments in multiscale free energy simulations

Physics-based free energy simulations enable the rigorous calculation of properties, such as conformational equilibria, solvation or binding free energies. While historically most applications have occurred at the atomistic level of resolution, a range of advances in the past years make it possible now to reliably cross the temporal, spatial and theory scales for the modeling of complex systems or the efficient prediction of results at the accuracy level of expensive quantum-mechanical calculations. In this mini-review, we discuss recent methodological advances as well as opportunities opened up by the introduction of machine learning approaches, which tackle the diverse challenges across the different scales, improve the accuracy and feasibility, and push the boundaries of multiscale free energy simulations.     

Probing light chain mutation effects on thrombin via molecular dynamics simulations and machine learning

Thrombin is a key component for chemotherapeutic and antithrombotic therapy development. As the physiologic and pathologic roles of the light chain still remain vague, here, we continue previous efforts to understand the impacts of the disease-associated single deletion of LYS9 in the light chain. By combining supervised and unsupervised machine learning methodologies and more traditional structural analyses on data from 10 μs molecular dynamics simulations, we show that the conformational ensemble of the ΔK9 mutant is significantly perturbed. Our analyses consistently indicate that LYS9 deletion destabilizes both the catalytic cleft and regulatory functional regions and result in some conformational changes that occur in tens to hundreds of nanosecond scaled motions. We also reveal that the two forms of thrombin each prefer a distinct binding mode of a Na⁺ ion. We expand our understanding of previous experimental observations and shed light on the mechanisms of the LYS9 deletion associated bleeding disorder by providing consistent but more quantitative and detailed structural analyses than early studies in literature. With a novel application of supervised learning, i.e. the decision tree learning on the hydrogen bonding features in the wild-type and ΔK9 mutant forms of thrombin, we predict that seven pairs of critical hydrogen bonding interactions are significant for establishing distinct behaviors of wild-type thrombin and its ΔK9 mutant form. Our calculations indicate the LYS9 in the light chain has both localized and long-range allosteric effects on thrombin, supporting the opinion that light chain has an important role as an allosteric effector.     

Applying machine learning based on multiscale classifiers to detect remote phenology patterns in Cerrado savanna trees

Plant phenology is one of the most reliable indicators of species responses to global climate change, motivating the development of new technologies for phenological monitoring. Digital cameras or near remote systems have been efficiently applied as multi-channel imaging sensors, where leaf color information is extracted from the RGB (Red, Green, and Blue) color channels, and the changes in green levels are used to infer leafing patterns of plant species. In this scenario, texture information is a great ally for image analysis that has been little used in phenology studies. We monitored leaf-changing patterns of Cerrado savanna vegetation by taking daily digital images. We extract RGB channels from the digital images and correlate them with phenological changes. Additionally, we benefit from the inclusion of textural metrics for quantifying spatial heterogeneity. Our first goals are: (1) to test if color change information is able to characterize the phenological pattern of a group of species; (2) to test if the temporal variation in image texture is useful to distinguish plant species; and (3) to test if individuals from the same species may be automatically identified using digital images. In this paper, we present a machine learning approach based on multiscale classifiers to detect phenological patterns in the digital images. Our results indicate that: (1) extreme hours (morning and afternoon) are the best for identifying plant species; (2) different plant species present a different behavior with respect to the color change information; and (3) texture variation along temporal images is promising information for capturing phenological patterns. Based on those results, we suggest that individuals from the same species and functional group might be identified using digital images, and introduce a new tool to help phenology experts in the identification of new individuals from the same species in the image and their location on the ground.     

Incremental learning in biological and machine learning systems

Incremental learning concepts are reviewed in machine learning and neurobiology. They are identified in evolution, neurodevelopment and learning. A timeline of qualitative axon, neuron and synapse development summarizes the review on neurodevelopment. A discussion of experimental results on data incremental learning with recurrent artificial neural networks reveals that incremental learning often seems to be more efficient or powerful than standard learning but can produce unexpected side effects. A characterization of incremental learning is proposed which takes the elaborated biological and machine learning concepts into account.     

Stable continual learning through structured multiscale plasticity manifolds

Biological plasticity is ubiquitous. How does the brain navigate this complex plasticity space, where any component can seemingly change, in adapting to an ever-changing environment? We build a systematic case that stable continuous learning is achieved by structured rules that enforce multiple, but not all, components to change together in specific directions. This rule-based low-dimensional plasticity manifold of permitted plasticity combinations emerges from cell type–specific molecular signaling and triggers cascading impacts that span multiple scales. These multiscale plasticity manifolds form the basis for behavioral learning and are dynamic entities that are altered by neuromodulation, metaplasticity, and pathology. We explore the strong links between heterogeneities, degeneracy, and plasticity manifolds and emphasize the need to incorporate plasticity manifolds into learning-theoretical frameworks and experimental designs.     

Melt-phase behavior of collapsed PMMA/PVC chains revealed by multiscale simulations

Single- and double-chain models of three stereoregular polymers, iso- and syndiotactic poly(methyl methacrylate) and isotactic poly(vinyl chloride), were extensively simulated using systematic coarse-grained (CG) potentials. It was found that, in vacuum, all of these long chains collapse in a two-stage process from their fully extended configurations into coils, and the two chains in each double-chain model ultimately become intertwined. Strong intermolecular interactions were found to occur between two chains of the same polymer (“like pairs”), which helps to explain the high densities of single-component melts. However, the intermolecular interactions between two chains of different polymers (“unlike pairs”) were stronger than those in like pairs. The enthalpy of mixing for unlike pairs—obtained from their intermolecular interaction energies—was negative, indicating that the two binary blends considered here are homogeneous systems. Moreover, a more negative enthalpy of mixing is suggested to correlate with better miscibility. These results agree well with corresponding experimental and simulated results, once again validating the accuracy of CG potentials when they are used to explore structural and energetic properties. The local structure captured by the isolated long chains dictates the ability to elucidate melt-phase behavior. A scheme involving the preparation of bulk models with initially collapsed chains was proposed; such CG models could be widely used to rapidly screen pairs of polymers for specific applications.

Open image in new window

Graphical Abstract Melt phase behaviors dictated by isolated chains

     

Linking 3D and 2D binding kinetics of membrane proteins by multiscale simulations

Membrane proteins are among the most functionally important proteins in cells. Unlike soluble proteins, they only possess two translational degrees of freedom on cell surfaces, and experience significant constraints on their rotations. As a result, it is currently challenging to characterize the in situ binding of membrane proteins. Using the membrane receptors CD2 and CD58 as a testing system, we developed a multiscale simulation framework to study the differences of protein binding kinetics between 3D and 2D environments. The association and dissociation processes were implemented by a coarse‐grained Monte‐Carlo algorithm, while the dynamic properties of proteins diffusing on lipid bilayer were captured from all‐atom molecular dynamic simulations. Our simulations show that molecular diffusion, linker flexibility and membrane fluctuations are important factors in adjusting binding kinetics. Moreover, by calibrating simulation parameters to the measurements of 3D binding, we derived the 2D binding constant which is quantitatively consistent with the experimental data, indicating that the method is able to capture the difference between 3D and 2D binding environments. Finally, we found that the 2D dissociation between CD2 and CD58 is about 100‐fold slower than the 3D dissociation. In summary, our simulation framework offered a generic approach to study binding mechanisms of membrane proteins.     

Uncertainty quantification in cerebral circulation simulations focusing on the collateral flow: Surrogate model approach with machine learning

Collateral circulation in the circle of Willis (CoW), closely associated with disease mechanisms and treatment outcomes, can be effectively investigated using one-dimensional–zero-dimensional hemodynamic simulations. As the entire cardiovascular system is considered in the simulation, it captures the systemic effects of local arterial changes, thus reproducing collateral circulation that reflects biological phenomena. The simulation facilitates rapid assessment of clinically relevant hemodynamic quantities under patient-specific conditions by incorporating clinical data. During patient-specific simulations, the impact of clinical data uncertainty on the simulated quantities should be quantified to obtain reliable results. However, as uncertainty quantification (UQ) is time-consuming and computationally expensive, its implementation in time-sensitive clinical applications is considered impractical. Therefore, we constructed a surrogate model based on machine learning using simulation data. The model accurately predicts the flow rate and pressure in the CoW in a few milliseconds. This reduced computation time enables the UQ execution with 100 000 predictions in a few minutes on a single CPU core and in less than a minute on a GPU. We performed UQ to predict the risk of cerebral hyperperfusion (CH), a life-threatening condition that can occur after carotid artery stenosis surgery if collateral circulation fails to function appropriately. We predicted the statistics of the postoperative flow rate increase in the CoW, which is a measure of CH, considering the uncertainties of arterial diameters, stenosis parameters, and flow rates measured using the patients’ clinical data. A sensitivity analysis was performed to clarify the impact of each uncertain parameter on the flow rate increase. Results indicated that CH occurred when two conditions were satisfied simultaneously: severe stenosis and when arteries of small diameter serve as the collateral pathway to the cerebral artery on the stenosis side. These findings elucidate the biological aspects of cerebral circulation in terms of the relationship between collateral flow and CH.     

Gaussian processes for machine learning

Gaussian processes (GPs) are natural generalisations of multivariate Gaussian random variables to infinite (countably or continuous) index sets. GPs have been applied in a large number of fields to a diverse range of ends, and very many deep theoretical analyses of various properties are available. This paper gives an introduction to Gaussian processes on a fairly elementary level with special emphasis on characteristics relevant in machine learning. It draws explicit connections to branches such as spline smoothing models and support vector machines in which similar ideas have been investigated. Gaussian process models are routinely used to solve hard machine learning problems. They are attractive because of their flexible non-parametric nature and computational simplicity. Treated within a Bayesian framework, very powerful statistical methods can be implemented which offer valid estimates of uncertainties in our predictions and generic model selection procedures cast as nonlinear optimization problems. Their main drawback of heavy computational scaling has recently been alleviated by the introduction of generic sparse approximations.13,78,31 The mathematical literature on GPs is large and often uses deep concepts which are not required to fully understand most machine learning applications. In this tutorial paper, we aim to present characteristics of GPs relevant to machine learning and to show up precise connections to other "kernel machines" popular in the community. Our focus is on a simple presentation, but references to more detailed sources are provided.     

Theoretical models and computer simulations of neural learning systems

It has been generally assumed for a long time that learning is accomplished in the central nervous system (CNS) by modifying strengths of ties between neurons. Various mechanisms may contribute to this process, but it is not known which are the specific mechanisms, and what are the rules by which they operate. Theoretical models, which are based on that general assumption are introduced. The purpose of the models is to suggest plausible ways by which learned information may be stored in the neural network, and be retrieved when it is needed. The networks in the models consist of four basic subunits, in accordance with identified units in the CNS: sensing, response, feeling, and control, plus association areas. The suggested operation rules are based on established operation rules of individual neurons, and assumed rules when neurons in groups are considered. Computer simulations are done, to check the consistency of the models, and to illustrate how they work. They simulate how an hypothetical kitten learns part of its environment, and show how relevant information may be stored and retrieved in its neuronal network. The suggested mechanisms could be examined in experiments, albeit not easy ones to conduct.     

Shark detection and classification with machine learning

Suitable shark conservation depends on well-informed population assessments. Direct methods such as scientific surveys and fisheries monitoring are adequate for defining population statuses, but species-specific indices of abundance and distribution coming from these sources are rare for most shark species. We can rapidly fill these information gaps by boosting media-based remote monitoring efforts with machine learning and automation.We created a database of 53,345 shark images covering 219 species of sharks, and packaged object-detection and image classification models into a Shark Detector bundle. The Shark Detector recognizes and classifies sharks from videos and images using transfer learning and convolutional neural networks (CNNs). We applied these models to common data-generation approaches of sharks: collecting occurrence records from photographs taken by the public or citizen scientists, processing baited remote camera footage and online videos, and data-mining Instagram. We examined the accuracy of each model and tested genus and species prediction correctness as a result of training data quantity.The Shark Detector can classify 47 species pertaining to 26 genera. It sorted heterogeneous datasets of images sourced from Instagram with 91% accuracy and classified species with 70% accuracy. It located sharks in baited remote footage and YouTube videos with 89% accuracy, and classified located subjects to the species level with 69% accuracy. All data-generation methods were processed without manual interaction.As media-based remote monitoring appears to dominate methods for observing sharks in nature, we developed an open-source Shark Detector to facilitate common identification applications. Prediction accuracy of the software pipeline increases as more images are added to the training dataset. We provide public access to the software on our GitHub page.