Learning Bayesian network classifiers using ant colony optimization

Bayesian networks are knowledge representation tools that model the (in)dependency relationships among variables for probabilistic reasoning. Classification with Bayesian networks aims to compute the class with the highest probability given a case. This special kind is referred to as Bayesian network classifiers. Since learning the Bayesian network structure from a dataset can be viewed as an optimization problem, heuristic search algorithms may be applied to build high-quality networks in medium- or large-scale problems, as exhaustive search is often feasible only for small problems. In this paper, we present our new algorithm, ABC-Miner, and propose several extensions to it. ABC-Miner uses ant colony optimization for learning the structure of Bayesian network classifiers. We report extended computational results comparing the performance of our algorithm with eight other classification algorithms, namely six variations of well-known Bayesian network classifiers, cAnt-Miner for discovering classification rules and a support vector machine algorithm.     

Towards a Mathematical Theory of Cortical Micro-circuits

The theoretical setting of hierarchical Bayesian inference is gaining acceptance as a framework for understanding cortical computation. In this paper, we describe how Bayesian belief propagation in a spatio-temporal hierarchical model, called Hierarchical Temporal Memory (HTM), can lead to a mathematical model for cortical circuits. An HTM node is abstracted using a coincidence detector and a mixture of Markov chains. Bayesian belief propagation equations for such an HTM node define a set of functional constraints for a neuronal implementation. Anatomical data provide a contrasting set of organizational constraints. The combination of these two constraints suggests a theoretically derived interpretation for many anatomical and physiological features and predicts several others. We describe the pattern recognition capabilities of HTM networks and demonstrate the application of the derived circuits for modeling the subjective contour effect. We also discuss how the theory and the circuit can be extended to explain cortical features that are not explained by the current model and describe testable predictions that can be derived from the model.     

Decision‐making under climatic uncertainty: A case study involving an Australian Ramsar‐listed wetland

Summary Conservation management in agricultural landscapes involves identification and prioritization of assets, and interventions to reverse or arrest decline. Planning requires synthesis of hydrological, ecological and agronomic information and intuitions. We provide a case study involving the Lake Warden Wetland System, a Ramsar‐listed site on the south coast of Western Australia threatened by salinity and flooding. As the relative merits of management options (including engineering‐based solutions and catchment revegetation) may be sensitive to climate change, we captured our knowledge and understanding of the effectiveness of options under different climate change scenarios using Bayesian belief networks. We insulated against overconfidence by an info‐gap analysis that describes the trade‐off between aspiration and immunity to uncertainty. Only engineering‐based solutions offer reasonable prospects for achieving stated conservation goals in the Lake Warden Wetland System within a 25‐year time horizon. Marginal gains derived from co‐investment in revegetation varied among the assets. We advocate explicit treatment of uncertainty and risk‐based approaches to decision‐making to equip managers with a means of progressing conservation goals. The complementary insights offered by Bayesian belief networks and info‐gap analysis provide a sound basis for managers to assess the extent to which candidate management actions are robust to uncertainty.     

Top-Down Feedback in an HMAX-Like Cortical Model of Object Perception Based on Hierarchical Bayesian Networks and Belief Propagation

Hierarchical generative models, such as Bayesian networks, and belief propagation have been shown to provide a theoretical framework that can account for perceptual processes, including feedforward recognition and feedback modulation. The framework explains both psychophysical and physiological experimental data and maps well onto the hierarchical distributed cortical anatomy. However, the complexity required to model cortical processes makes inference, even using approximate methods, very computationally expensive. Thus, existing object perception models based on this approach are typically limited to tree-structured networks with no loops, use small toy examples or fail to account for certain perceptual aspects such as invariance to transformations or feedback reconstruction. In this study we develop a Bayesian network with an architecture similar to that of HMAX, a biologically-inspired hierarchical model of object recognition, and use loopy belief propagation to approximate the model operations (selectivity and invariance). Crucially, the resulting Bayesian network extends the functionality of HMAX by including top-down recursive feedback. Thus, the proposed model not only achieves successful feedforward recognition invariant to noise, occlusions, and changes in position and size, but is also able to reproduce modulatory effects such as illusory contour completion and attention. Our novel and rigorous methodology covers key aspects such as learning using a layerwise greedy algorithm, combining feedback information from multiple parents and reducing the number of operations required. Overall, this work extends an established model of object recognition to include high-level feedback modulation, based on state-of-the-art probabilistic approaches. The methodology employed, consistent with evidence from the visual cortex, can be potentially generalized to build models of hierarchical perceptual organization that include top-down and bottom-up interactions, for example, in other sensory modalities.     

Integrating multi-attribute similarity networks for robust representation of the protein space

MOTIVATION: A global view of the protein space is essential for functional and evolutionary analysis of proteins. In order to achieve this, a similarity network can be built using pairwise relationships among proteins. However, existing similarity networks employ a single similarity measure and therefore their utility depends highly on the quality of the selected measure. A more robust representation of the protein space can be realized if multiple sources of information are used. RESULTS: We propose a novel approach for analyzing multi-attribute similarity networks by combining random walks on graphs with Bayesian theory. A multi-attribute network is created by combining sequence and structure based similarity measures. For each attribute of the similarity network, one can compute a measure of affinity from a given protein to every other protein in the network using random walks. This process makes use of the implicit clustering information of the similarity network, and we show that it is superior to naive, local ranking methods. We then combine the computed affinities using a Bayesian framework. In particular, when we train a Bayesian model for automated classification of a novel protein, we achieve high classification accuracy and outperform single attribute networks. In addition, we demonstrate the effectiveness of our technique by comparison with a competing kernel-based information integration approach.     

Comparison of statistical methods for identification of Streptococcus thermophilus, Enterococcus faecalis, and Enterococcus faecium from randomly amplified polymorphic DNA patterns

Thermophilic streptococci play an important role in the manufacture of many European cheeses, and a rapid and reliable method for their identification is needed. Randomly amplified polymorphic DNA (RAPD) PCR (RAPD-PCR) with two different primers coupled to hierarchical cluster analysis has proven to be a powerful tool for the classification and typing of Streptococcus thermophilus, Enterococcus faecium, and Enterococcus faecalis (G. Moschetti, G. Blaiotta, M. Aponte, P. Catzeddu, F. Villani, P. Deiana, and S. Coppola, J. Appl. Microbiol. 85:25-36, 1998). In order to develop a fast and inexpensive method for the identification of thermophilic streptococci, RAPD-PCR patterns were generated with a single primer (XD9), and the results were analyzed using artificial neural networks (Multilayer Perceptron, Radial Basis Function network, and Bayesian network) and multivariate statistical techniques (cluster analysis, linear discriminant analysis, and classification trees). Cluster analysis allowed the identification of S. thermophilus but not of enterococci. A Bayesian network proved to be more effective than a Multilayer Perceptron or a Radial Basis Function network for the identification of S. thermophilus, E. faecium, and E. faecalis using simplified RAPD-PCR patterns (obtained by summing the bands in selected areas of the patterns). The Bayesian network also significantly outperformed two multivariate statistical techniques (linear discriminant analysis and classification trees) and proved to be less sensitive to the size of the training set and more robust in the response to patterns belonging to unknown species.     

Applying a Novel Combination of Techniques to Develop a Predictive Model for Diabetes Complications

Among the many related issues of diabetes management, its complications constitute the main part of the heavy burden of this disease. The aim of this paper is to develop a risk advisor model to predict the chances of diabetes complications according to the changes in risk factors. As the starting point, an inclusive list of (k) diabetes complications and (n) their correlated predisposing factors are derived from the existing endocrinology text books. A type of data meta-analysis has been done to extract and combine the numeric value of the relationships between these two. The whole n (risk factors) - k (complications) model was broken down into k different (n-1) relationships and these (n-1) dependencies were broken into n (1-1) models. Applying regression analysis (seven patterns) and artificial neural networks (ANN), we created models to show the (1-1) correspondence between factors and complications. Then all 1-1 models related to an individual complication were integrated using the naïve Bayes theorem. Finally, a Bayesian belief network was developed to show the influence of all risk factors and complications on each other. We assessed the predictive power of the 1-1 models by R², F-ratio and adjusted R² equations; sensitivity, specificity and positive predictive value were calculated to evaluate the final model using real patient data. The results suggest that the best fitted regression models outperform the predictive ability of an ANN model, as well as six other regression patterns for all 1-1 models.     

A copula method for modeling directional dependence of genes

Background  

Genes interact with each other as basic building blocks of life, forming a complicated network. The relationship between groups of genes with different functions can be represented as gene networks. With the deposition of huge microarray data sets in public domains, study on gene networking is now possible. In recent years, there has been an increasing interest in the reconstruction of gene networks from gene expression data. Recent work includes linear models, Boolean network models, and Bayesian networks. Among them, Bayesian networks seem to be the most effective in constructing gene networks. A major problem with the Bayesian network approach is the excessive computational time. This problem is due to the interactive feature of the method that requires large search space. Since fitting a model by using the copulas does not require iterations, elicitation of the priors, and complicated calculations of posterior distributions, the need for reference to extensive search spaces can be eliminated leading to manageable computational affords. Bayesian network approach produces a discretely expression of conditional probabilities. Discreteness of the characteristics is not required in the copula approach which involves use of uniform representation of the continuous random variables. Our method is able to overcome the limitation of Bayesian network method for gene-gene interaction, i.e. information loss due to binary transformation.     

面向异步多时延基因调控网络建模的高阶动态贝叶斯网络模型及其结构 学习算法

目的：由基因芯片数据精确学习建模具有异步多时延表达调控关系的基因调控网络。方法：提出了一种高阶动态贝叶斯网 络模型,并给出了网络结构学习算法,该模型假定基因的调控过程为多阶马尔科夫过程,从而能够建模基因调控网络中的异步多 时延特性。结果：由酵母基因调控网络一个子网络人工生成了加入10%含噪声的表达数据用于调控网络结构学习。在75%的后验 概率下,本文提出的高阶动态贝叶斯网络模型能够正确建模实际网络中全部的异步多时延调控关系,而经典动态贝叶斯网络仅 能够正确建模实际网络中1/3的调控关系;ROC曲线对比表明在各个后验概率水平上高阶动态贝叶斯网络模型的效果均优于经 典动态贝叶斯网络。结论：本文提出的高阶动态贝叶斯网络模型能够精确学习建模具有异步多时延表达调控关系的基因调控网 络。     

面向异步多时延基因调控网络建模的高阶动态贝叶斯网络模型及其结构学习算法

目的：由基因芯片数据精确学习建模具有异步多时延表达调控关系的基因调控网络。方法：提出了一种高阶动态贝叶斯网络模型,并给出了网络结构学习算法,该模型假定基因的调控过程为多阶马尔科夫过程,从而能够建模基因调控网络中的异步多时延特性。结果：由酵母基因调控网络一个子网络人工生成了加入10%含噪声的表达数据用于调控网络结构学习。在75%的后验概率下,本文提出的高阶动态贝叶斯网络模型能够正确建模实际网络中全部的异步多时延调控关系,而经典动态贝叶斯网络仅能够正确建模实际网络中1/3的调控关系;ROC曲线对比表明在各个后验概率水平上高阶动态贝叶斯网络模型的效果均优于经典动态贝叶斯网络。结论：本文提出的高阶动态贝叶斯网络模型能够精确学习建模具有异步多时延表达调控关系的基因调控网络。     

Causal inference in biomolecular pathways using a Bayesian network approach and an Implicit method

We introduce here the concept of Implicit networks which provide, like Bayesian networks, a graphical modelling framework that encodes the joint probability distribution for a set of random variables within a directed acyclic graph. We show that Implicit networks, when used in conjunction with appropriate statistical techniques, are very attractive for their ability to understand and analyze biological data. Particularly, we consider here the use of Implicit networks for causal inference in biomolecular pathways. In such pathways, an Implicit network encodes dependencies among variables (proteins, genes), can be trained to learn causal relationships (regulation, interaction) between them and then used to predict the biological response given the status of some key proteins or genes in the network. We show that Implicit networks offer efficient methodologies for learning from observations without prior knowledge and thus provide a good alternative to classical inference in Bayesian networks when priors are missing. We illustrate our approach by an application to simulated data for a simplified signal transduction pathway of the epidermal growth factor receptor (EGFR) protein.     

The Use of Bayesian Networks to Assess the Quality of Evidence from Research Synthesis: 1.

Background

The grades of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in systematic reviews, health technology assessment and guideline development organisations throughout the world. A key advantage to this approach is that it aids transparency regarding judgments on the quality of evidence. However, the intricacies of making judgments about research methodology and evidence make the GRADE system complex and challenging to apply without training.

Methods

We have developed a semi-automated quality assessment tool (SAQAT) l based on GRADE. This is informed by responses by reviewers to checklist questions regarding characteristics that may lead to unreliability. These responses are then entered into the Bayesian network to ascertain the probabilities of risk of bias, inconsistency, indirectness, imprecision and publication bias conditional on review characteristics. The model then combines these probabilities to provide a probability for each of the GRADE overall quality categories. We tested the model using a range of plausible scenarios that guideline developers or review authors could encounter.

Results

Overall, the model reproduced GRADE judgements for a range of scenarios. Potential advantages over standard assessment are use of explicit and consistent weightings for different review characteristics, forcing consideration of important but sometimes neglected characteristics and principled downgrading where small but important probabilities of downgrading are accrued across domains.

Conclusions

Bayesian networks have considerable potential for use as tools to assess the validity of research evidence. The key strength of such networks lies in the provision of a statistically coherent method for combining probabilities across a complex framework based on both belief and evidence. In addition to providing tools for less experienced users to implement reliability assessment, the potential for sensitivity analyses and automation may be beneficial for application and the methodological development of reliability tools.     

Artificial intelligence-guided analysis of cytologic data

A design for the integration of artificial intelligence (AI) technology with large databases of clinical and objective cytologic data, such as are on file at the University of Chicago, is presented. Among the key features of this approach are the use of a knowledge representation structure based upon an associative network, the use of a Bayesian belief network as a method of managing uncertainty in the system, and the use of neural networks and unsupervised learning algorithms as a means of discovering patterns within this database. Such an automated approach is necessary, given the complexity and interdependence of these data, to gain an understanding of their dependence structure and to assist in their exploration and analysis.     

Uncovering Community Structures with Initialized Bayesian Nonnegative Matrix Factorization

Uncovering community structures is important for understanding networks. Currently, several nonnegative matrix factorization algorithms have been proposed for discovering community structure in complex networks. However, these algorithms exhibit some drawbacks, such as unstable results and inefficient running times. In view of the problems, a novel approach that utilizes an initialized Bayesian nonnegative matrix factorization model for determining community membership is proposed. First, based on singular value decomposition, we obtain simple initialized matrix factorizations from approximate decompositions of the complex network’s adjacency matrix. Then, within a few iterations, the final matrix factorizations are achieved by the Bayesian nonnegative matrix factorization method with the initialized matrix factorizations. Thus, the network’s community structure can be determined by judging the classification of nodes with a final matrix factor. Experimental results show that the proposed method is highly accurate and offers competitive performance to that of the state-of-the-art methods even though it is not designed for the purpose of modularity maximization.     

miniTUBA: medical inference by network integration of temporal data using Bayesian analysis

MOTIVATION: Many biomedical and clinical research problems involve discovering causal relationships between observations gathered from temporal events. Dynamic Bayesian networks are a powerful modeling approach to describe causal or apparently causal relationships, and support complex medical inference, such as future response prediction, automated learning, and rational decision making. Although many engines exist for creating Bayesian networks, most require a local installation and significant data manipulation to be practical for a general biologist or clinician. No software pipeline currently exists for interpretation and inference of dynamic Bayesian networks learned from biomedical and clinical data. RESULTS: miniTUBA is a web-based modeling system that allows clinical and biomedical researchers to perform complex medical/clinical inference and prediction using dynamic Bayesian network analysis with temporal datasets. The software allows users to choose different analysis parameters (e.g. Markov lags and prior topology), and continuously update their data and refine their results. miniTUBA can make temporal predictions to suggest interventions based on an automated learning process pipeline using all data provided. Preliminary tests using synthetic data and laboratory research data indicate that miniTUBA accurately identifies regulatory network structures from temporal data. AVAILABILITY: miniTUBA is available at http://www.minituba.org.     

Bayesian belief networks to analyse trade-offs among ecosystem services at the regional scale

Knowledge on trade-offs and synergies among ecosystem services is crucial for the design of land use strategies that optimize ecosystem service delivery. Correlation coefficients, obtained through pairwise comparison of ecosystem service provision maps, have been put forward as suitable indicators to quantify these interactions. However, for more in depth analyses of trade-offs and synergies where driving forces of interactions need to be determined, more sophisticated methods are needed. Although Bayesian belief networks have been frequently mentioned as promising tools to investigate interactions among ecosystem services, up till now, no structured approaches to do so have been suggested. This paper presents a way to analyse trade-offs and synergies among ecosystem services together with their driving forces. Joint probability distributions of ecosystem service pairs, which can be calculated by using Bayesian belief network models, are used to quantify interactions. The paper demonstrates the approach by quantifying trade-offs and synergies among several ecosystem services in Flanders, Belgium. Our analysis identifies two bundles of ecosystem services which react synergistically. Wood production and several regulating services on the one hand and food production and soil formation on the other hand. Trade-offs are identified among food production and most of the other services that were included into the analysis. In addition to these general findings, the analysis shows that the identified interactions may change depending on the considered environmental conditions, specified through soil type, land cover and land use.     

Use of unsupervised and supervised artificial neural networks for the identification of lactic acid bacteria on the basis of SDS-PAGE patterns of whole cell proteins

Conventional multivariate statistical techniques (hierarchical cluster analysis, linear discriminant analysis) and unsupervised (Kohonen Self Organizing Map) and supervised (Bayesian network) artificial neural networks were compared for as tools for the classification and identification of 352 SDS-PAGE patterns of whole cell proteins of lactic acid bacteria belonging to 22 species of the genera Lactobacillus, Leuconostoc, Enterococcus, Lactococcus and Streptococcus including 47 reference strains. Electrophoretic data were pre-treated using the logistic weighting function described by Piraino et al. [Piraino, P., Ricciardi, A., Lanorte, M. T., Malkhazova, I., Parente, E., 2002. A new procedure for data reduction in electrophoretic fingerprints of whole-cell proteins. Biotechnol. Lett. 24, 1477-1482]. Hierarchical cluster analysis provided a satisfactory classification of the patterns but was unable to discriminate some species (Leuconostoc, Lb. sakei/Lb. curvatus, Lb. acidophilus/Lb. helveticus, Lb. plantarum/Lb. paraplantarum, Lc. lactis/Lc. raffinolactis). A 7x7 Kohonen self-organizing map (KSOM), trained with the patterns of the reference strains, provided a satisfactory classification of the patterns and was able to discriminate more species than hierarchical cluster analysis. The map was used in predictive mode to identify unknown strains and provided results which in 85.5% of cases matched the classification obtained by hierarchical cluster analysis. Two supervised tools, linear discriminant analysis and a 23:5:2 Bayesian network were proven to be highly effective in the discrimination of SDS-PAGE patterns of Lc. lactis from those of other species. We conclude that data reduction by logistic weighting coupled to traditional multivariate statistical analysis or artificial neural networks provide an effective tool for the classification and identification of lactic acid bacteria on the basis of SDS-PAGE patterns of whole cell proteins.     

Gene network inference using continuous time Bayesian networks: a comparative study and application to Th17 cell differentiation

Background

Dynamic aspects of gene regulatory networks are typically investigated by measuring system variables at multiple time points. Current state-of-the-art computational approaches for reconstructing gene networks directly build on such data, making a strong assumption that the system evolves in a synchronous fashion at fixed points in time. However, nowadays omics data are being generated with increasing time course granularity. Thus, modellers now have the possibility to represent the system as evolving in continuous time and to improve the models’ expressiveness.

Results

Continuous time Bayesian networks are proposed as a new approach for gene network reconstruction from time course expression data. Their performance was compared to two state-of-the-art methods: dynamic Bayesian networks and Granger causality analysis. On simulated data, the methods comparison was carried out for networks of increasing size, for measurements taken at different time granularity densities and for measurements unevenly spaced over time. Continuous time Bayesian networks outperformed the other methods in terms of the accuracy of regulatory interactions learnt from data for all network sizes. Furthermore, their performance degraded smoothly as the size of the network increased. Continuous time Bayesian networks were significantly better than dynamic Bayesian networks for all time granularities tested and better than Granger causality for dense time series. Both continuous time Bayesian networks and Granger causality performed robustly for unevenly spaced time series, with no significant loss of performance compared to the evenly spaced case, while the same did not hold true for dynamic Bayesian networks. The comparison included the IRMA experimental datasets which confirmed the effectiveness of the proposed method. Continuous time Bayesian networks were then applied to elucidate the regulatory mechanisms controlling murine T helper 17 (Th17) cell differentiation and were found to be effective in discovering well-known regulatory mechanisms, as well as new plausible biological insights.

Conclusions

Continuous time Bayesian networks were effective on networks of both small and large size and were particularly feasible when the measurements were not evenly distributed over time. Reconstruction of the murine Th17 cell differentiation network using continuous time Bayesian networks revealed several autocrine loops, suggesting that Th17 cells may be auto regulating their own differentiation process.     

Comparison of classification methods for detecting associations between SNPs and chick mortality

Multi-category classification methods were used to detect SNP-mortality associations in broilers. The objective was to select a subset of whole genome SNPs associated with chick mortality. This was done by categorizing mortality rates and using a filter-wrapper feature selection procedure in each of the classification methods evaluated. Different numbers of categories (2, 3, 4, 5 and 10) and three classification algorithms (naïve Bayes classifiers, Bayesian networks and neural networks) were compared, using early and late chick mortality rates in low and high hygiene environments. Evaluation of SNPs selected by each classification method was done by predicted residual sum of squares and a significance test-related metric. A naïve Bayes classifier, coupled with discretization into two or three categories generated the SNP subset with greatest predictive ability. Further, an alternative categorization scheme, which used only two extreme portions of the empirical distribution of mortality rates, was considered. This scheme selected SNPs with greater predictive ability than those chosen by the methods described previously. Use of extreme samples seems to enhance the ability of feature selection procedures to select influential SNPs in genetic association studies.