A sensitivity analysis of the volumetric spatial decomposition algorithm

Recently, we proposed a new computational approach for the volumetric spatial decomposition of a three-dimensional bone structure into its basic rod and plate elements. This method was based on an image skeletonization approach, where two model parameters were used to identify an ideal skeleton. The goal of this study was to estimate the sensitivity of local morphometric indices to these two model parameters. Our results showed that the rod derived indices behaved more smoothly than plate derived indices, which suggests that rod derived indices are more trustworthy. The results also demonstrated that it was reasonable to reduce the model to only one parameter by setting the noise parameter n to twice the value of the slenderness parameter s. In conclusion, we found that local morphometric indices are reliable measures showing large differences between samples and thus may shed new light on structural differences of trabecular bone in a local fashion by adequately choosing one single optimization parameter.     

A mathematical model for selective differentiation of neural progenitor cells on micropatterned polymer substrates

The biological hypothesis that the astrocyte-secreted cytokine, interleukin-6 (IL6), stimulates differentiation of adult rat hippocampal progenitor cells (AHPCs) is considered from a mathematical perspective. The proposed mathematical model includes two different mechanisms for stimulation and is based on mass-action kinetics. Both biological mechanisms involve sequential binding, with one pathway solely utilizing surface receptors while the other pathway also involves soluble receptors. Choosing biologically-reasonable values for parameters, simulations of the mathematical model show good agreement with experimental results. A global sensitivity analysis is also conducted to determine both the most influential and non-influential parameters on cellular differentiation, providing additional insights into the biological mechanisms.     

Defining and detecting structural sensitivity in biological models: developing a new framework

When we construct mathematical models to represent biological systems, there is always uncertainty with regards to the model specification—whether with respect to the parameters or to the formulation of model functions. Sometimes choosing two different functions with close shapes in a model can result in substantially different model predictions: a phenomenon known in the literature as structural sensitivity, which is a significant obstacle to improving the predictive power of biological models. In this paper, we revisit the general definition of structural sensitivity, compare several more specific definitions and discuss their usefulness for the construction and analysis of biological models. Then we propose a general approach to reveal structural sensitivity with regards to certain system properties, which considers infinite-dimensional neighbourhoods of the model functions: a far more powerful technique than the conventional approach of varying parameters for a fixed functional form. In particular, we suggest a rigorous method to unearth sensitivity with respect to the local stability of systems’ equilibrium points. We present a method for specifying the neighbourhood of a general unknown function with \(n\) inflection points in terms of a finite number of local function properties, and provide a rigorous proof of its completeness. Using this powerful result, we implement our method to explore sensitivity in several well-known multicomponent ecological models and demonstrate the existence of structural sensitivity in these models. Finally, we argue that structural sensitivity is an important intrinsic property of biological models, and a direct consequence of the complexity of the underlying real systems.     

Polydesigns and causal inference

In an increasingly common class of studies, the goal is to evaluate causal effects of treatments that are only partially controlled by the investigator. In such studies there are two conflicting features: (1) a model on the full cohort design and data can identify the causal effects of interest, but can be sensitive to extreme regions of that design's data, where model specification can have more impact; and (2) models on a reduced design (i.e., a subset of the full data), for example, conditional likelihood on matched subsets of data, can avoid such sensitivity, but do not generally identify the causal effects. We propose a framework to assess how inference is sensitive to designs by exploring combinations of both the full and reduced designs. We show that using such a "polydesign" framework generates a rich class of methods that can identify causal effects and that can also be more robust to model specification than methods using only the full design. We discuss implementation of polydesign methods, and provide an illustration in the evaluation of a needle exchange program.     

Functional rarefaction: estimating functional diversity from field data

Studies in biodiversity-ecosystem function and conservation biology have led to the development of diversity indices that take species' functional differences into account. We identify two broad classes of indices: those that monotonically increase with species richness (MSR indices) and those that weight the contribution of each species by abundance or occurrence (weighted indices). We argue that weighted indices are easier to estimate without bias but tend to ignore information provided by rare species. Conversely, MSR indices fully incorporate information provided by rare species but are nearly always underestimated when communities are not exhaustively surveyed. This is because of the well-studied fact that additional sampling of a community may reveal previously undiscovered species. We use the rarefaction technique from species richness studies to address sample-size-induced bias when estimating functional diversity indices. Rarefaction transforms any given MSR index into a family of unbiased weighted indices, each with a different level of sensitivity to rare species. Thus rarefaction simultaneously solves the problem of bias and the problem of sensitivity to rare species. We present formulae and algorithms for conducting a functional rarefaction analysis of the two most widely cited MSR indices: functional attribute diversity (FAD) and Petchey and Gaston's functional diversity (FD). These formulae also demonstrate a relationship between three seemingly unrelated functional diversity indices: FAD, FD and Rao's quadratic entropy. Statistical theory is also provided in order to prove that all desirable statistical properties of species richness rarefaction are preserved for functional rarefaction.     

Application of global sensitivity analysis to determine goals for design of experiments: an example study on antibody-producing cell cultures

Global sensitivity analysis (GSA) can be used to quantify the importance of model parameters and their interactions with respect to model output. In this study, the Sobol' method for GSA is applied to a dynamic model of monoclonal antibody-producing mammalian cell cultures in order to identify the parameters that need to be accurately determined experimentally. Our results show that most parameters have low sensitivity indices and exhibit strong interactions with one another. These parameters can be set at their nominal values and unnecessary experimentation can therefore be avoided. In contrast, certain parameters are identified as sensitive, necessitating their estimation given sufficiently rich experimental data. Moreover, parameter sensitivity varies during culture time in a biologically meaningful manner. In conclusion, GSA can serve as an excellent precursor to optimal experiment design.     

An in silico erythropoiesis model rationalizing synergism between stem cell factor and erythropoietin

Stem cell factor (SCF) and erythropoietin (EPO) are two most recognized growth factors that play in concert to control in vitro erythropoiesis. However, exact mechanisms underlying the interplay of these growth factors in vitro remain unclear. We developed a mathematical model to study co-signaling effects of SCF and EPO utilizing the ERK1/2 and GATA-1 pathways (activated by SCF and EPO) that drive the proliferation and differentiation of erythroid progenitors. The model was simplified and formulated based on three key features: synergistic contribution of SCF and EPO on ERK1/2 activation, positive feedback effects on proliferation and differentiation, and cross-inhibition effects of activated ERK1/2 and GATA-1. The model characteristics were developed to correspond with biological observations made known thus far. Our simulation suggested that activated GATA-1 has a more dominant cross-inhibition effect and stronger positive feedback response on differentiation than the proliferation pathway, while SCF contributed more to the activation of ERK1/2 than EPO. A sensitivity analysis performed to gauge the dynamics of the system was able to identify the most sensitive model parameters and illustrated a contribution of transient activity in EPO ligand to growth factor synergism. Based on theoretical arguments, we have successfully developed a model that can simulate growth factor synergism observed in vitro for erythropoiesis. This hypothesized model can be applied to further computational studies in biological systems where synergistic effects of two ligands are seen.     

Biomarkers as Predictors in Health and Ecological Risk Assessment

Biomarkers are measurable biological parameters that change in response to xenobiotic exposure and other environmental or physiological stressors, and can be indices of toxicant exposure or effects. If the biomarkers are sufficiently specific and well characterized, they can have great utility in the risk assessment process by providing an indication of the degree of exposure of humans or animals in natural populations to a specific xenobiotic or class of xenobiotics. Most biomarkers are effective as indices of exposure, but adequate information is rarely available on the appropriate dose-response curves to have well-described biomarkers of effect that can be widely applicable to additional populations. Specific examples of acetylcholinest-erase inhibition following exposure to organophosphorus insecticides are cited from experiments in both mammals (rats) and fish. These experiments have indicated that the degree of inhibition can be readily influenced by endogenous (e.g., age) and exogenous (e.g., chemical exposures) factors, and that the degree of inhibition is not readily correlated with toxicological effects. Caution is urged, therefore, in an attempt to utilize biomarkers in the risk assessment process until more complete documentation is available on the specificity, sensitivity, and time course of changes, and on the impact of multiple exposures or the time of exposures.     

Image-Based Assessment of Growth and Signaling Changes in Cancer Cells Mediated by Direct Cell-Cell Contact

Background

Many important biological processes are controlled through cell-cell interactions, including the colonization of metastatic tumor cells and the control of differentiation of stem cells within their niche. Despite the crucial importance of the cellular environment in regulating cellular signaling, in vitro methods for the study of such interactions are difficult and/or indirect.

Methodology/Principal Findings

We report on the development of an image-based method for distinguishing two cell types grown in coculture. Furthermore, cells of one type that are in direct contact with cells of a second type (adjacent cells) can be analyzed separately from cells that are not within a single well. Changes are evaluated using population statistics, which are useful in detecting subtle changes across two populations. We have used this system to characterize changes in the LNCaP prostate carcinoma cell line when grown in contact with human vascular endothelial cells (HUVECs). We find that the expression and phosphorylation of WWOX is reduced in LNCaP cells when grown in direct contact with HUVECs. Reduced WWOX signaling has been associated with reduced activation or expression of JNK and p73. We find that p73 levels are also reduced in LNCaP cells grown in contact with HUVECs, but we did not observe such a change in JNK levels.

Conclusions/Significance

We find that the method described is statistically robust and can be adapted to a wide variety of studies where cell function or signaling are affected by heterotypic cell-cell contact. Ironically, a potential challenge to the method is its high level of sensitivity is capable of classifying events as statistically significant (due to the high number cells evaluated individually), when the biological effect may be less clear. The methodology would be best used in conjunction with additional methods to evaluate the biological role of potentially subtle differences between populations. However, many important events, such as the establishment of a metastatic tumor, occur through rare but important changes, and methods such as we describe here can be used to identify and characterize the contribution of the environment to these changes.     

A combination of SILAC and nucleotide acyl phosphate labelling reveals unexpected targets of the Rsk inhibitor BI-D1870

Protein kinase inhibitors frequently have interesting effects that cannot be fully ascribed to the intended target kinase(s) but identifying additional targets that might explain the effects is not straightforward. By comparing two different inhibitors of the Rsk (p90 ribosomal S6 kinase) kinases, we found that the increasingly used compound BI-D1870 had biological effects in murine DCs (dendritic cells) that could not be solely ascribed to Rsk or other documented targets. We assessed the ability of BI-D1870 and a second Rsk inhibitor, BIX 02565 to protect enzyme active sites from reaction with biotinylated nucleotide acyl phosphates. Using SILAC (stable isotope labelling by amino acids in cell culture)-labelled DC lysates as a source of enzyme targets, we identify several kinases that interact with BI-D1870 but not with BIX 02565. We confirmed that these kinases, including Slk, Lok and Mst1, are inhibited by BI-D1870 but to a much lesser extent by BIX 02565 and that phosphorylation of some of their substrates is blocked by BI-D1870 in living cells. Our results suggest that the BI-D1870 inhibitor should be used with caution. The SILAC-based methodology we used should be useful for further comparative unbiased profiling of the target spectrum of kinase inhibitors with interesting biological effects under conditions that closely mimic those found in cells.     

An integrated framework for building trustworthy data-driven epidemiological models: Application to the COVID-19 outbreak in New York City

Epidemiological models can provide the dynamic evolution of a pandemic but they are based on many assumptions and parameters that have to be adjusted over the time the pandemic lasts. However, often the available data are not sufficient to identify the model parameters and hence infer the unobserved dynamics. Here, we develop a general framework for building a trustworthy data-driven epidemiological model, consisting of a workflow that integrates data acquisition and event timeline, model development, identifiability analysis, sensitivity analysis, model calibration, model robustness analysis, and projection with uncertainties in different scenarios. In particular, we apply this framework to propose a modified susceptible–exposed–infectious–recovered (SEIR) model, including new compartments and model vaccination in order to project the transmission dynamics of COVID-19 in New York City (NYC). We find that we can uniquely estimate the model parameters and accurately project the daily new infection cases, hospitalizations, and deaths, in agreement with the available data from NYC’s government’s website. In addition, we employ the calibrated data-driven model to study the effects of vaccination and timing of reopening indoor dining in NYC.     

Parameter estimation and sensitivity analysis in an agent-based model of Leishmania major infection

Computer models of disease take a systems biology approach toward understanding host-pathogen interactions. In particular, data driven computer model calibration is the basis for inference of immunological and pathogen parameters, assessment of model validity, and comparison between alternative models of immune or pathogen behavior. In this paper we describe the calibration and analysis of an agent-based model of Leishmania major infection. A model of macrophage loss following uptake of necrotic tissue is proposed to explain macrophage depletion following peak infection. Using Gaussian processes to approximate the computer code, we perform a sensitivity analysis to identify important parameters and to characterize their influence on the simulated infection. The analysis indicates that increasing growth rate can favor or suppress pathogen loads, depending on the infection stage and the pathogen's ability to avoid detection. Subsequent calibration of the model against previously published biological observations suggests that L. major has a relatively slow growth rate and can replicate for an extended period of time before damaging the host cell.     

Surrogate variable analysis using partial least squares (SVA-PLS) in gene expression studies

MOTIVATION: In a typical gene expression profiling study, our prime objective is to identify the genes that are differentially expressed between the samples from two different tissue types. Commonly, standard analysis of variance (ANOVA)/regression is implemented to identify the relative effects of these genes over the two types of samples from their respective arrays of expression levels. But, this technique becomes fundamentally flawed when there are unaccounted sources of variability in these arrays (latent variables attributable to different biological, environmental or other factors relevant in the context). These factors distort the true picture of differential gene expression between the two tissue types and introduce spurious signals of expression heterogeneity. As a result, many genes which are actually differentially expressed are not detected, whereas many others are falsely identified as positives. Moreover, these distortions can be different for different genes. Thus, it is also not possible to get rid of these variations by simple array normalizations. This both-way error can lead to a serious loss in sensitivity and specificity, thereby causing a severe inefficiency in the underlying multiple testing problem. In this work, we attempt to identify the hidden effects of the underlying latent factors in a gene expression profiling study by partial least squares (PLS) and apply ANCOVA technique with the PLS-identified signatures of these hidden effects as covariates, in order to identify the genes that are truly differentially expressed between the two concerned tissue types. RESULTS: We compare the performance of our method SVA-PLS with standard ANOVA and a relatively recent technique of surrogate variable analysis (SVA), on a wide variety of simulation settings (incorporating different effects of the hidden variable, under situations with varying signal intensities and gene groupings). In all settings, our method yields the highest sensitivity while maintaining relatively reasonable values for the specificity, false discovery rate and false non-discovery rate. Application of our method to gene expression profiling for acute megakaryoblastic leukemia shows that our method detects an additional six genes, that are missed by both the standard ANOVA method as well as SVA, but may be relevant to this disease, as can be seen from mining the existing literature.     

Method of predicting reference condition biota affects the performance and interpretation of ecological indices

1. The statistical rigour and interpretability of ecological assessments is strongly affected by how well we predict the biological assemblages expected to occur in the absence of human‐caused stress, i.e. the reference condition. In this study, we examined how the specific method used to predict the reference condition affected the performance of two commonly used types of ecological index: RIVPACS‐based O/E indices and multimetric indices (MMIs). 2. These two types of index have generally relied on different approaches to predicting the reference condition. For MMIs, some type of regionalisation is typically used to describe the range of metric values among reference sites and hence the expected range at assessed sites. For O/E indices, continuous modelling is used to predict how the biota varies among sites both among and within regions. Because the prediction method differs for these two types of index, it has been impossible to judge if differences in index performance (accuracy, precision, responsiveness and sensitivity) are caused by differences in the way reference condition biota are predicted or by differences in what the indices measure. 3. We used a common data set of 94 reference sites and 255 managed sites and the same potential set of predictor variables to compare the performance of five different MMIs and three O/E indices that were derived from different prediction methods: null models, multiple linear regression (MLR), classification and regression trees, Random Forests (RF) and linear discriminant functions models (LDM). We then calculated values of these indices for samples collected from the managed catchments as well as samples collected from 13 reference sites that were progressively altered in known ways by a simulation programme. 4. Both the type of predictor used and the type of index affected overall index performance. Modelled indices generally had the greatest sensitivity in assessing managed sites as biologically different from reference. Index sensitivity was determined by both an aspect of index precision (10th percentile of reference condition values) and responsiveness. The O/E indices showed the best scope of response to known biological alteration. All three O/E indices decreased linearly in response to simulated alteration in both overall assemblage structure (Bray‐Curtis dissimilarity) and taxa loss. The MMIs declined linearly from low to intermediate levels of assemblage alteration but were less responsive between intermediate and high levels of biological alteration. 5. Insights gained from simulations can aid in testing assumptions regarding index response to stress and help ensure that we select indices that are ecologically interpretable and most useful to resource managers.     

Model sensitivity and use of the comparative finite element method in mammalian jaw mechanics: mandible performance in the gray wolf

Finite Element Analysis (FEA) is a powerful tool gaining use in studies of biological form and function. This method is particularly conducive to studies of extinct and fossilized organisms, as models can be assigned properties that approximate living tissues. In disciplines where model validation is difficult or impossible, the choice of model parameters and their effects on the results become increasingly important, especially in comparing outputs to infer function. To evaluate the extent to which performance measures are affected by initial model input, we tested the sensitivity of bite force, strain energy, and stress to changes in seven parameters that are required in testing craniodental function with FEA. Simulations were performed on FE models of a Gray Wolf (Canis lupus) mandible. Results showed that unilateral bite force outputs are least affected by the relative ratios of the balancing and working muscles, but only ratios above 0.5 provided balancing-working side joint reaction force relationships that are consistent with experimental data. The constraints modeled at the bite point had the greatest effect on bite force output, but the most appropriate constraint may depend on the study question. Strain energy is least affected by variation in bite point constraint, but larger variations in strain energy values are observed in models with different number of tetrahedral elements, masticatory muscle ratios and muscle subgroups present, and number of material properties. These findings indicate that performance measures are differentially affected by variation in initial model parameters. In the absence of validated input values, FE models can nevertheless provide robust comparisons if these parameters are standardized within a given study to minimize variation that arise during the model-building process. Sensitivity tests incorporated into the study design not only aid in the interpretation of simulation results, but can also provide additional insights on form and function.     

Young-of-the-year fish assemblages as indicators of anthropogenic disturbances in large tributaries of the Seine River Basin (France)

A variety of indices targeting a number of different biological assemblages have been developed to assess aquatic ecosystem condition, identify the drivers of alteration and provide information on possible restoration measures. Fish-based indices, commonly focused on adult assemblages, are capable of indicating declines in condition, but often provide limited information on the ultimate causes for the observed changes. Using young-of-the-year (YOY) fish assemblages has been suggested as a means of improving the sensitivity of fish-based indicators. To verify this assumption, we first model, using boosted regression trees, the occurrence of 16 YOY fish species as a function of twenty two environmental factors in 227 reference (i.e., least disturbed) habitats located in the river Seine (France) and its two main tributaries (i.e., Oise and Marne rivers). We then validate the species models using an independent dataset of 74 reference habitats. Finally, using three independent data sets reflecting different categories of disturbances (i.e., physico-chemical disturbances, physical disturbances induced by navigation and a mix of both disturbances), we measure the deviation between expected and observed species occurrences to evaluate our species models ability in detecting these disturbances. The models are ecologically meaningful and overall perform well in discriminating between reference and disturbed habitats, showing a stable response for all unimpaired habitats and highlighting the main habitat disturbances tested.     

Regression Analysis for Constraining Free Parameters in Electrophysiological Models of Cardiac Cells

A major challenge in computational biology is constraining free parameters in mathematical models. Adjusting a parameter to make a given model output more realistic sometimes has unexpected and undesirable effects on other model behaviors. Here, we extend a regression-based method for parameter sensitivity analysis and show that a straightforward procedure can uniquely define most ionic conductances in a well-known model of the human ventricular myocyte. The model''s parameter sensitivity was analyzed by randomizing ionic conductances, running repeated simulations to measure physiological outputs, then collecting the randomized parameters and simulation results as “input” and “output” matrices, respectively. Multivariable regression derived a matrix whose elements indicate how changes in conductances influence model outputs. We show here that if the number of linearly-independent outputs equals the number of inputs, the regression matrix can be inverted. This is significant, because it implies that the inverted matrix can specify the ionic conductances that are required to generate a particular combination of model outputs. Applying this idea to the myocyte model tested, we found that most ionic conductances could be specified with precision (R² > 0.77 for 12 out of 16 parameters). We also applied this method to a test case of changes in electrophysiology caused by heart failure and found that changes in most parameters could be well predicted. We complemented our findings using a Bayesian approach to demonstrate that model parameters cannot be specified using limited outputs, but they can be successfully constrained if multiple outputs are considered. Our results place on a solid mathematical footing the intuition-based procedure simultaneously matching a model''s output to several data sets. More generally, this method shows promise as a tool to define model parameters, in electrophysiology and in other biological fields.     

Dynamic Bayesian sensitivity analysis of a myocardial metabolic model

Dynamic compartmentalized metabolic models are identified by a large number of parameters, several of which are either non-physical or extremely difficult to measure. Typically, the available data and prior information is insufficient to fully identify the system. Since the models are used to predict the behavior of unobserved quantities, it is important to understand how sensitive the output of the system is to perturbations in the poorly identifiable parameters. Classically, it is the goal of sensitivity analysis to asses how much the output changes as a function of the parameters. In the case of dynamic models, the output is a function of time and therefore its sensitivity is a time dependent function. If the output is a differentiable function of the parameters, the sensitivity at one time instance can be computed from its partial derivatives with respect to the parameters. The time course of these partial derivatives describes how the sensitivity varies in time.When the model is not uniquely identifiable, or if the solution of the parameter identification problem is known only approximately, we may have not one, but a distribution of possible parameter values. This is always the case when the parameter identification problem is solved in a statistical framework. In that setting, the proper way to perform sensitivity analysis is to not rely on the values of the sensitivity functions corresponding to a single model, but to consider the distributed nature of the sensitivity functions, inherited from the distribution of the vector of the model parameters.In this paper we propose a methodology for analyzing the sensitivity of dynamic metabolic models which takes into account the variability of the sensitivity over time and across a sample. More specifically, we draw a representative sample from the posterior density of the vector of model parameters, viewed as a random variable. To interpret the output of this doubly varying sensitivity analysis, we propose visualization modalities particularly effective at displaying simultaneously variations over time and across a sample. We perform an analysis of the sensitivity of the concentrations of lactate and glycogen in cytosol, and of ATP, ADP, NAD⁺ and NADH in cytosol and mitochondria, to the parameters identifying a three compartment model for myocardial metabolism during ischemia.     

Neuronal firing sensitivity to morphologic and active membrane parameters

Both the excitability of a neuron's membrane, driven by active ion channels, and dendritic morphology contribute to neuronal firing dynamics, but the relative importance and interactions between these features remain poorly understood. Recent modeling studies have shown that different combinations of active conductances can evoke similar firing patterns, but have neglected how morphology might contribute to homeostasis. Parameterizing the morphology of a cylindrical dendrite, we introduce a novel application of mathematical sensitivity analysis that quantifies how dendritic length, diameter, and surface area influence neuronal firing, and compares these effects directly against those of active parameters. The method was applied to a model of neurons from goldfish Area II. These neurons exhibit, and likely contribute to, persistent activity in eye velocity storage, a simple model of working memory. We introduce sensitivity landscapes, defined by local sensitivity analyses of firing rate and gain to each parameter, performed globally across the parameter space. Principal directions over which sensitivity to all parameters varied most revealed intrinsic currents that most controlled model output. We found domains where different groups of parameters had the highest sensitivities, suggesting that interactions within each group shaped firing behaviors within each specific domain. Application of our method, and its characterization of which models were sensitive to general morphologic features, will lead to advances in understanding how realistic morphology participates in functional homeostasis. Significantly, we can predict which active conductances, and how many of them, will compensate for a given age- or development-related structural change, or will offset a morphologic perturbation resulting from trauma or neurodegenerative disorder, to restore normal function. Our method can be adapted to analyze any computational model. Thus, sensitivity landscapes, and the quantitative predictions they provide, can give new insight into mechanisms of homeostasis in any biological system.