Analysis of an optimal hidden Markov model for secondary structure prediction

Background  

Secondary structure prediction is a useful first step toward 3D structure prediction. A number of successful secondary structure prediction methods use neural networks, but unfortunately, neural networks are not intuitively interpretable. On the contrary, hidden Markov models are graphical interpretable models. Moreover, they have been successfully used in many bioinformatic applications. Because they offer a strong statistical background and allow model interpretation, we propose a method based on hidden Markov models.     

Explainable deep transfer learning model for disease risk prediction using high-dimensional genomic data

Building an accurate disease risk prediction model is an essential step in the modern quest for precision medicine. While high-dimensional genomic data provides valuable data resources for the investigations of disease risk, their huge amount of noise and complex relationships between predictors and outcomes have brought tremendous analytical challenges. Deep learning model is the state-of-the-art methods for many prediction tasks, and it is a promising framework for the analysis of genomic data. However, deep learning models generally suffer from the curse of dimensionality and the lack of biological interpretability, both of which have greatly limited their applications. In this work, we have developed a deep neural network (DNN) based prediction modeling framework. We first proposed a group-wise feature importance score for feature selection, where genes harboring genetic variants with both linear and non-linear effects are efficiently detected. We then designed an explainable transfer-learning based DNN method, which can directly incorporate information from feature selection and accurately capture complex predictive effects. The proposed DNN-framework is biologically interpretable, as it is built based on the selected predictive genes. It is also computationally efficient and can be applied to genome-wide data. Through extensive simulations and real data analyses, we have demonstrated that our proposed method can not only efficiently detect predictive features, but also accurately predict disease risk, as compared to many existing methods.     

Widespread allele-specific topological domains in the human genome are not confined to imprinted gene clusters

Neural networks such as variational autoencoders (VAE) perform dimensionality reduction for the visualization and analysis of genomic data, but are limited in their interpretability: it is unknown which data features are represented by each embedding dimension. We present siVAE, a VAE that is interpretable by design, thereby enhancing downstream analysis tasks. Through interpretation, siVAE also identifies gene modules and hubs without explicit gene network inference. We use siVAE to identify gene modules whose connectivity is associated with diverse phenotypes such as iPSC neuronal differentiation efficiency and dementia, showcasing the wide applicability of interpretable generative models for genomic data analysis.

     

深度学习在药物活性预测研究中的应用

药物从研发到临床应用需要耗费较长的时间，研发期间的投入成本可高达十几亿元。而随着医药研发与人工智能的结合以及生物信息学的飞速发展，药物活性相关数据急剧增加，传统的实验手段进行药物活性预测已经难以满足药物研发的需求。借助算法来辅助药物研发，解决药物研发中的各种问题能够大大推动药物研发进程。传统机器学习方法尤其是随机森林、支持向量机和人工神经网络在药物活性方面能够达到较高的预测精度。深度学习由于具有多层神经网络，模型可以接收高维的输入变量且不需要人工限定数据输入特征，可以拟合较为复杂的函数模型，应用于药物研发可以进一步提高各个环节的效率。在药物活性预测中应用较为广泛的深度学习模型主要是深度神经网络（deep neural networks，DNN）、循环神经网络（recurrent neural networks，RNN）和自编码器（auto encoder，AE），而生成对抗网络（generative adversarial networks，GAN）由于其生成数据的能力常常被用来和其他模型结合进行数据增强。近年来深度学习在药物分子活性预测方面的研究和应用综述表明，深度学习模型的准确度和效率均高于传统实验方法和传统机器学习方法。因此，深度学习模型有望成为药物研发领域未来十年最重要的辅助计算模型。     

Bayesian nonparametric quantile process regression and estimation of marginal quantile effects

Flexible estimation of multiple conditional quantiles is of interest in numerous applications, such as studying the effect of pregnancy-related factors on low and high birth weight. We propose a Bayesian nonparametric method to simultaneously estimate noncrossing, nonlinear quantile curves. We expand the conditional distribution function of the response in I-spline basis functions where the covariate-dependent coefficients are modeled using neural networks. By leveraging the approximation power of splines and neural networks, our model can approximate any continuous quantile function. Compared to existing models, our model estimates all rather than a finite subset of quantiles, scales well to high dimensions, and accounts for estimation uncertainty. While the model is arbitrarily flexible, interpretable marginal quantile effects are estimated using accumulative local effect plots and variable importance measures. A simulation study shows that our model can better recover quantiles of the response distribution when the data are sparse, and an analysis of birth weight data is presented.     

应用神经网络和多元回归技术预测森林产量

应用传统统计技术常会因样本小和测量数据不符某种分布而受到限制。本文评价一种前馈型神经网络算法以预测落叶阔叶林产量。另外,还介绍一种由定性变为定量的数据变换方法,以用相对小的样本建立多元回归预测模型。数据变换方法有助于改善多元回归模型的预测效果。在本实验的条件下,研究结果表明神经网络技术能够产生最好的预测效果.     

Applications of proportional odds ordinal logistic regression models and continuation ratio models in examining the association of physical inactivity with erectile dysfunction among type 2 diabetic patients

[Purpose]Many studies have observed a high prevalence of erectile dysfunction among individuals performing physical activity in less leisure-time. However, this relationship in patients with type 2 diabetic patients is not well studied. In exposure outcome studies with ordinal outcome variables, investigators often try to make the outcome variable dichotomous and lose information by collapsing categories. Several statistical models have been developed to make full use of all information in ordinal response data, but they have not been widely used in public health research. In this paper, we discuss the application of two statistical models to determine the association of physical inactivity with erectile dysfunction among patients with type 2 diabetes.[Methods]A total of 204 married men aged 20-60 years with a diagnosis of type 2 diabetes at the outpatient unit of the Department of Endocrinology at PSG hospitals during the months of May and June 2019 were studied. We examined the association between physical inactivity and erectile dysfunction using proportional odds ordinal logistic regression models and continuation ratio models.[Results]The proportional odds model revealed that patients with diabetes who perform leisure time physical activity for over 40 minutes per day have reduced odds of erectile dysfunction (odds ratio=0.38) across the severity categories of erectile dysfunction after adjusting for age and duration of diabetes.[Conclusion]The present study suggests that physical inactivity has a negative impact on erectile function. We observed that the simple logistic regression model had only 75% efficiency compared to the proportional odds model used here; hence, more valid estimates were obtained here.     

Panel semiparametric quantile regression neural network for electricity consumption forecasting

Addressing the forecasting issues is one of the core objectives of developing and restructuring of electric power industry in China. However, there are not enough efforts that have been made to develop an accurate electricity consumption forecasting procedure. In this paper, a panel semiparametric quantile regression neural network (PSQRNN) is developed by combining an artificial neural network and semiparametric quantile regression for panel data. By embedding penalized quantile regression with least absolute shrinkage and selection operator (LASSO), ridge regression and backpropagation, PSQRNN keeps the flexibility of nonparametric models and the interpretability of parametric models simultaneously. The prediction accuracy is evaluated based on China's electricity consumption data set, and the results indicate that PSQRNN performs better compared with three benchmark methods including BP neural network (BP), Support Vector Machine (SVM) and Quantile Regression Neural Network (QRNN).     

Graph neural network approaches for drug-target interactions

Developing new drugs remains prohibitively expensive, time-consuming, and often involves safety issues. Accurate prediction of drug-target interactions (DTIs) can guide the drug discovery process and thus facilitate drug development. Non-Euclidian data such as drug-like molecule structures, key pocket residue structures, and protein interaction networks can be represented effectively using graphs. Therefore, the emerging graph neural network has been rapidly applied to predict DTIs, and proved effective in finding repositioning drugs and accelerating drug discovery. In this review, we provide a brief overview of deep neural networks used in DTI models. Then, we summarize the database required for DTI prediction, followed by a comprehensive introduction of applications of graph neural networks for DTI prediction. We also highlight current challenges and future directions to guide the further development of this field.     

Simultaneous modeling of multivariate heterogeneous responses and heteroskedasticity via a two-stage composite likelihood

Multivariate heterogeneous responses and heteroskedasticity have attracted increasing attention in recent years. In genome-wide association studies, effective simultaneous modeling of multiple phenotypes would improve statistical power and interpretability. However, a flexible common modeling system for heterogeneous data types can pose computational difficulties. Here we build upon a previous method for multivariate probit estimation using a two-stage composite likelihood that exhibits favorable computational time while retaining attractive parameter estimation properties. We extend this approach to incorporate multivariate responses of heterogeneous data types (binary and continuous), and possible heteroskedasticity. Although the approach has wide applications, it would be particularly useful for genomics, precision medicine, or individual biomedical prediction. Using a genomics example, we explore statistical power and confirm that the approach performs well for hypothesis testing and coverage percentages under a wide variety of settings. The approach has the potential to better leverage genomics data and provide interpretable inference for pleiotropy, in which a locus is associated with multiple traits.     

Including network knowledge into Cox regression models for biomarker signature discovery

Discovery of prognostic and diagnostic biomarker gene signatures for diseases, such as cancer, is seen as a major step toward a better personalized medicine. During the last decade various methods have been proposed for that purpose. However, one important obstacle for making gene signatures a standard tool in clinical diagnosis is the typical low reproducibility of these signatures combined with the difficulty to achieve a clear biological interpretation. For that purpose in the last years there has been a growing interest in approaches that try to integrate information from molecular interaction networks. Most of these methods focus on classification problems, that is learn a model from data that discriminates patients into distinct clinical groups. Far less has been published on approaches that predict a patient's event risk. In this paper, we investigate eight methods that integrate network information into multivariable Cox proportional hazard models for risk prediction in breast cancer. We compare the prediction performance of our tested algorithms via cross‐validation as well as across different datasets. In addition, we highlight the stability and interpretability of obtained gene signatures. In conclusion, we find GeneRank‐based filtering to be a simple, computationally cheap and highly predictive technique to integrate network information into event time prediction models. Signatures derived via this method are highly reproducible.     

Selective prediction-set models with coverage rate guarantees

The current approach to using machine learning (ML) algorithms in healthcare is to either require clinician oversight for every use case or use their predictions without any human oversight. We explore a middle ground that lets ML algorithms abstain from making a prediction to simultaneously improve their reliability and reduce the burden placed on human experts. To this end, we present a general penalized loss minimization framework for training selective prediction-set (SPS) models, which choose to either output a prediction set or abstain. The resulting models abstain when the outcome is difficult to predict accurately, such as on subjects who are too different from the training data, and achieve higher accuracy on those they do give predictions for. We then introduce a model-agnostic, statistical inference procedure for the coverage rate of an SPS model that ensembles individual models trained using K-fold cross-validation. We find that SPS ensembles attain prediction-set coverage rates closer to the nominal level and have narrower confidence intervals for its marginal coverage rate. We apply our method to train neural networks that abstain more for out-of-sample images on the MNIST digit prediction task and achieve higher predictive accuracy for ICU patients compared to existing approaches.     

Extensive feature detection of N-terminal protein sorting signals

MOTIVATION: The prediction of localization sites of various proteins is an important and challenging problem in the field of molecular biology. TargetP, by Emanuelsson et al. (J. Mol. Biol., 300, 1005-1016, 2000) is a neural network based system which is currently the best predictor in the literature for N-terminal sorting signals. One drawback of neural networks, however, is that it is generally difficult to understand and interpret how and why they make such predictions. In this paper, we aim to generate simple and interpretable rules as predictors, and still achieve a practical prediction accuracy. We adopt an approach which consists of an extensive search for simple rules and various attributes which is partially guided by human intuition. RESULTS: We have succeeded in finding rules whose prediction accuracies come close to that of TargetP, while still retaining a very simple and interpretable form. We also discuss and interpret the discovered rules.     

Identifying Populations at High Risk for Occupational Back Injury with Neutral Networks

For this study a simulation is conducted to investigate the accuracy of neural networks and logistic regression in identifying populations at high risk for occupational back injury. In contrast to most standard regression techniques, neural networks do not rely on linearity or explicitly specifying the nature of the association. Because the underlying relationships between work exposures, personal risk factors, and injury are often not well defined, neural networks may prove useful for injury risk assessment. Accuracy was assessed by comparing the injury status to the predicted level of risk in each worker. In simulations of a non-linear association, workers (used in the training data) were correctly classified 85% of the time with neural networks, 74% of the time with the main effects logistic model, and 79% of the time with the fully-specified logistic model. Using the test data, however, workers were correctly classified 67% of the time with neural networks, and 71% and 69% of the time with the main effects and fully specified logistic models, respectively. Simulations of a null association indicated that neural networks may be more likely to overfit random associations. These findings provide a valuable guide concerning statistical methodology for identifying high-risk worker populations.     

Neural networks for clustered and longitudinal data using mixed effects models

Although most statistical methods for the analysis of longitudinal data have focused on retrospective models of association, new advances in mobile health data have presented opportunities for predicting future health status by leveraging an individual's behavioral history alongside data from similar patients. Methods that incorporate both individual-level and sample-level effects are critical to using these data to its full predictive capacity. Neural networks are powerful tools for prediction, but many assume input observations are independent even when they are clustered or correlated in some way, such as in longitudinal data. Generalized linear mixed models (GLMM) provide a flexible framework for modeling longitudinal data but have poor predictive power particularly when the data are highly nonlinear. We propose a generalized neural network mixed model that replaces the linear fixed effect in a GLMM with the output of a feed-forward neural network. The model simultaneously accounts for the correlation structure and complex nonlinear relationship between input variables and outcomes, and it utilizes the predictive power of neural networks. We apply this approach to predict depression and anxiety levels of schizophrenic patients using longitudinal data collected from passive smartphone sensor data.     

Bayesian multiple index models for environmental mixtures

An important goal of environmental health research is to assess the risk posed by mixtures of environmental exposures. Two popular classes of models for mixtures analyses are response-surface methods and exposure-index methods. Response-surface methods estimate high-dimensional surfaces and are thus highly flexible but difficult to interpret. In contrast, exposure-index methods decompose coefficients from a linear model into an overall mixture effect and individual index weights; these models yield easily interpretable effect estimates and efficient inferences when model assumptions hold, but, like most parsimonious models, incur bias when these assumptions do not hold. In this paper, we propose a Bayesian multiple index model framework that combines the strengths of each, allowing for non-linear and non-additive relationships between exposure indices and a health outcome, while reducing the dimensionality of the exposure vector and estimating index weights with variable selection. This framework contains response-surface and exposure-index models as special cases, thereby unifying the two analysis strategies. This unification increases the range of models possible for analysing environmental mixtures and health, allowing one to select an appropriate analysis from a spectrum of models varying in flexibility and interpretability. In an analysis of the association between telomere length and 18 organic pollutants in the National Health and Nutrition Examination Survey (NHANES), the proposed approach fits the data as well as more complex response-surface methods and yields more interpretable results.     

环境微生物研究中机器学习算法及应用

微生物在环境中无处不在，它们不仅是生物地球化学循环和环境演化的关键参与者，也在环境监测、生态治理和保护中发挥着重要作用。随着高通量技术的发展，大量微生物数据产生，运用机器学习对环境微生物大数据进行建模和分析，在微生物标志物识别、污染物预测和环境质量预测等领域的科学研究和社会应用方面均具有重要意义。机器学习可分为监督学习和无监督学习2大类。在微生物组学研究当中，无监督学习通过聚类、降维等方法高效地学习输入数据的特征，进而对微生物数据进行整合和归类。监督学习运用有特征和标记的微生物数据集训练模型，在面对只有特征没有标记的数据时可以判断出标记，从而实现对新数据的分类、识别和预测。然而，复杂的机器学习算法通常以牺牲可解释性为代价来重点关注模型预测的准确性。机器学习模型通常可以看作预测特定结果的“黑匣子”，即对模型如何得出预测所知甚少。为了将机器学习更多地运用于微生物组学研究、提高我们提取有价值的微生物信息的能力，深入了解机器学习算法、提高模型的可解释性尤为重要。本文主要介绍在环境微生物领域常用的机器学习算法和基于微生物组数据的机器学习模型的构建步骤，包括特征选择、算法选择、模型构建和评估等，并对各种机器学习模型在环境微生物领域的应用进行综述，深入探究微生物组与周围环境之间的关联，探讨提高模型可解释性的方法，并为未来环境监测、环境健康预测提供科学参考。     

PASNet: pathway-associated sparse deep neural network for prognosis prediction from high-throughput data

Background

Predicting prognosis in patients from large-scale genomic data is a fundamentally challenging problem in genomic medicine. However, the prognosis still remains poor in many diseases. The poor prognosis may be caused by high complexity of biological systems, where multiple biological components and their hierarchical relationships are involved. Moreover, it is challenging to develop robust computational solutions with high-dimension, low-sample size data.

Results

In this study, we propose a Pathway-Associated Sparse Deep Neural Network (PASNet) that not only predicts patients’ prognoses but also describes complex biological processes regarding biological pathways for prognosis. PASNet models a multilayered, hierarchical biological system of genes and pathways to predict clinical outcomes by leveraging deep learning. The sparse solution of PASNet provides the capability of model interpretability that most conventional fully-connected neural networks lack. We applied PASNet for long-term survival prediction in Glioblastoma multiforme (GBM), which is a primary brain cancer that shows poor prognostic performance. The predictive performance of PASNet was evaluated with multiple cross-validation experiments. PASNet showed a higher Area Under the Curve (AUC) and F1-score than previous long-term survival prediction classifiers, and the significance of PASNet’s performance was assessed by Wilcoxon signed-rank test. Furthermore, the biological pathways, found in PASNet, were referred to as significant pathways in GBM in previous biology and medicine research.

Conclusions

PASNet can describe the different biological systems of clinical outcomes for prognostic prediction as well as predicting prognosis more accurately than the current state-of-the-art methods. PASNet is the first pathway-based deep neural network that represents hierarchical representations of genes and pathways and their nonlinear effects, to the best of our knowledge. Additionally, PASNet would be promising due to its flexible model representation and interpretability, embodying the strengths of deep learning. The open-source code of PASNet is available at https://github.com/DataX-JieHao/PASNet.

     

Braun‐Blanquet's legacy and data analysis in vegetation science

Abstract. This article investigates whether the Braun‐Blanquet abundance/dominance (AD) scores that commonly appear in phytosociological tables can properly be analysed by conventional multivariate analysis methods such as Principal Components Analysis and Correspondence Analysis. The answer is a definite NO. The source of problems is that the AD values express species performance on a scale, namely the ordinal scale, on which differences are not interpretable. There are several arguments suggesting that no matter which methods have been preferred in contemporary numerical syntaxonomy and why, ordinal data should be treated in an ordinal way. In addition to the inadmissibility of arithmetic operations with the AD scores, these arguments include interpretability of dissimilarities derived from ordinal data, consistency of all steps throughout the analysis and universality of the method which enables simultaneous treatment of various measurement scales. All the ordination methods that are commonly used, for example, Principal Components Analysis and all variants of Correspondence Analysis as well as standard cluster analyses such as Ward's method and group average clustering, are inappropriate when using AD data. Therefore, the application of ordinal clustering and scaling methods to traditional phytosociological data is advocated. Dissimilarities between relevés should be calculated using ordinal measures of resemblance, and ordination and clustering algorithms should also be ordinal in nature. A good ordination example is Non‐metric Multidimensional Scaling (NMDS) as long as it is calculated from an ordinal dissimilarity measure such as the Goodman & Kruskal γ coefficient, and for clustering the new OrdClAn‐H and OrdClAn‐N methods.     

Empirical modelling of lake water-temperature relationships: a comparison of approaches

1.  As a result of the role that temperature plays in many aquatic processes, good predictive models of annual maximum near-surface lake water temperature across large spatial scales are needed, particularly given concerns regarding climate change. Comparisons of suitable modelling approaches are required to determine their relative merit and suitability for providing good predictions of current conditions. We developed models predicting annual maximum near-surface lake water temperatures for lakes across Canada using four statistical approaches: multiple regression, regression tree, artificial neural networks and Bayesian multiple regression.
2.  Annual maximum near-surface (from 0 to 2 m) lake water-temperature data were obtained for more than 13 000 lakes and were matched to geographic, climatic, lake morphology, physical habitat and water chemistry data. We modelled 2348 lakes and three subsets thereof encompassing different spatial scales and predictor variables to identify the relative importance of these variables at predicting lake temperature.
3.  Although artificial neural networks were marginally better for three of the four data sets, multiple regression was considered to provide the best solution based on the combination of model performance and computational complexity. Climatic variables and date of sampling were the most important variables for predicting water temperature in our models.
4.  Lake morphology did not play a substantial role in predicting lake temperature across any of the spatial scales. Maximum near-surface temperatures for Canadian lakes appeared to be dominated by large-scale climatic and geographic patterns, rather than lake-specific variables, such as lake morphology and water chemistry.