P > .05: The incorrect interpretation of “not significant” results is a significant problem

Statistically nonsignificant (p > .05) results from a null hypothesis significance test (NHST) are often mistakenly interpreted as evidence that the null hypothesis is true—that there is “no effect” or “no difference.” However, many of these results occur because the study had low statistical power to detect an effect. Power below 50% is common, in which case a result of no statistical significance is more likely to be incorrect than correct. The inference of “no effect” is not valid even if power is high. NHST assumes that the null hypothesis is true; p is the probability of the data under the assumption that there is no effect. A statistical test cannot confirm what it assumes. These incorrect statistical inferences could be eliminated if decisions based on p values were replaced by a biological evaluation of effect sizes and their confidence intervals. For a single study, the observed effect size is the best estimate of the population effect size, regardless of the p value. Unlike p values, confidence intervals provide information about the precision of the observed effect. In the biomedical and pharmacology literature, methods have been developed to evaluate whether effects are “equivalent,” rather than zero, as tested with NHST. These methods could be used by biological anthropologists to evaluate the presence or absence of meaningful biological effects. Most of what appears to be known about no difference or no effect between sexes, between populations, between treatments, and other circumstances in the biological anthropology literature is based on invalid statistical inference.     

Human sleep under the influence of pulsed radiofrequency electromagnetic fields: A polysomnographic study using standardized conditions

To investigate the influence of radiofrequency electromagnetic fields (EMFs) of cellular phone GSM signals on human sleep electroencephalographic (EEG) pattern, all-night polysomnographies of 24 healthy male subjects were recorded, both with and without exposure to a circular polarized EMF (900 MHz, pulsed with a frequency of 217 Hz, pulse width 577 μs, power flux density 0.2 W/m². Suppression of rapid eye movement (REM) sleep as well as a sleep-inducing effect under field exposure did not reach statistical significance, so that previous results indicating alterations of these sleep parameters could not be replicated. Spectral power analysis also did not reveal any alterations of the EEG rhythms during EMF exposure. The failure to confirm our previous results might be due to dose-dependent effects of the EMF on the human sleep profile. Bioelectromagnetics 19:199–202, 1998. © 1998 Wiley-Liss, Inc.     

Nonlinear dynamical law governs magnetic field induced changes in lymphoid phenotype.

The results of many different types of animal and human studies dealing with the biological effects of exposure to low frequency electromagnetic fields (EMFs) have consistently been both positive and negative. We addressed the question of why this pattern had occurred so commonly in biological studies involving exposure to EMFs and hypothesized that it stemmed from the prevalent use of a linear model to characterize what are inherently nonlinear input-output relationships. The hypothesis was tested by analyzing biological data using a novel statistical procedure that could be adjusted to detect either nonlinear or linear effects. The reliability of the procedure was established using positive and negative controls and by comparison with the results obtained from sampling a known nonlinear system. In four independent experiments, male and female mice were exposed continuously to 0.1 or 0.5 mT, 60 Hz, for 175 days, and the effect on 20 immune parameters was measured using flow cytometry and functional assays. In each experiment, EMF exposure resulted in statistically significant changes in lymphoid phenotype when and only when the response of the animals to the fields was analyzed as if it were governed by nonlinear laws. Our results suggest that the pattern of inconsistency in the EMF bioeffects studies is an artifact resulting from an incorrect choice of the conceptual model for the relation between the field and the biological effect it causally determines.     

POLYTOMIES AND THE POWER OF PHYLOGENETIC INFERENCE

Although phylogenetic hypotheses can provide insights into mechanisms of evolution, their utility is limited by our inability to differentiate simultaneous speciation events (hard polytomies) from rapid cladogenesis (soft polytomies). In the present paper, we tested the potential for statistical power analysis to differentiate between hard and soft polytomies in molecular phytogenies. Classical power analysis typically is used a priori to determine the sample size required to detect a particular effect size at a particular level of significance (a) with a certain power (1 – β). A posteriori, power analysis is used to infer whether failure to reject a null hypothesis results from lack of an effect or from insufficient data (i.e., low power). We adapted this approach to molecular data to infer whether polytomies result from simultaneous branching events or from insufficient sequence information. We then used this approach to determine the amount of sequence data (sample size) required to detect a positive branch length (effect size). A worked example is provided based on the auklets (Charadriiformes: Alcidae), a group of seabirds among which relationships are represented by a polytomy, despite analyses of over 3000 bp of sequence data. We demonstrate the calculation of effect sizes and sample sizes from sequence data using a normal curve test for difference of a proportion from an expected value and a t-test for a difference of a mean from an expected value. Power analyses indicated that the data for the auklets should be sufficient to differentiate speciation events that occurred at least 100,000 yr apart (the duration of the shortest glacial and interglacial events of the Pleistocene), 2.6 million years ago.     

Nonlinear response of the immune system to power-frequency magnetic fields

Studies of the effects of power-frequency electromagnetic fields (EMFs) on the immune and other body systems produced positive and negative results, and this pattern was usually interpreted to indicate the absence of real effects. However, if the biological effects of EMFs were governed by nonlinear laws, deterministic responses to fields could occur that were both real and inconsistent, thereby leading to both types of results. The hypothesis of real inconsistent effects due to EMFs was tested by exposing mice to 1 G, 60 Hz for 1-105 days and observing the effect on 20 immune parameters, using flow cytometry and functional assays. The data were evaluated by means of a novel statistical procedure that avoided averaging away oppositely directed changes in different animals, which we perceived to be the problem in some of the earlier EMF studies. The reliability of the procedure was shown using appropriate controls. In three independent experiments involving exposure for 21 or more days, the field altered lymphoid phenotype even though the changes in individual immune parameters were inconsistent. When the data were evaluated using traditional linear statistical methods, no significant difference in any immune parameter was found. We were able to mimic the results by sampling from known chaotic systems, suggesting that deterministic chaos could explain the effect of fields on the immune system. We conclude that exposure to power-frequency fields produced changes in the immune system that were both real and inconsistent.     

Estimating effect sizes of differentially expressed genes for power and sample-size assessments in microarray experiments

Summary In microarray screening for differentially expressed genes using multiple testing, assessment of power or sample size is of particular importance to ensure that few relevant genes are removed from further consideration prematurely. In this assessment, adequate estimation of the effect sizes of differentially expressed genes is crucial because of its substantial impact on power and sample‐size estimates. However, conventional methods using top genes with largest observed effect sizes would be subject to overestimation due to random variation. In this article, we propose a simple estimation method based on hierarchical mixture models with a nonparametric prior distribution to accommodate random variation and possible large diversity of effect sizes across differential genes, separated from nuisance, nondifferential genes. Based on empirical Bayes estimates of effect sizes, the power and false discovery rate (FDR) can be estimated to monitor them simultaneously in gene screening. We also propose a power index that concerns selection of top genes with largest effect sizes, called partial power. This new power index could provide a practical compromise for the difficulty in achieving high levels of usual overall power as confronted in many microarray experiments. Applications to two real datasets from cancer clinical studies are provided.     

Experimental model for ELF-EMF exposure: Concern for human health

Low frequency (LF) electromagnetic fields (EMFs) are abundantly present in modern society and in the last 20 years the interest about the possible effect of extremely low frequency (ELF) EMFs on human health has increased progressively. Epidemiological studies, designed to verify whether EMF exposure may be a potential risk factor for health, have led to controversial results. The possible association between EMFs and an increased incidence of childhood leukemia, brain tumors or neurodegenerative diseases was not fully elucidated. On the other hand, EMFs are widely used, in neurology, psychiatry, rheumatology, orthopedics and dermatology, both in diagnosis and in therapy.In vitro studies may help to evaluate the mechanism by which LF-EMFs affect biological systems.Invitro model of wound healing used keratinocytes (HaCaT), neuroblastoma cell line (SH-SY5Y) as a model for analysis of differentiation, metabolism and functions related to neurodegenerative processes, and monocytic cell line (THP-1) was used as a model for inflammation and cytokines production, while leukemic cell line (K562) was used as a model for hematopoietic differentiation.MCP-1, a chemokine that regulates the migration and infiltration of memory T cells, natural killer (NK), monocytes and epithelial cells, has been demonstrated to be induced and involved in various diseases.Since, varying the parameters of EMFs different effects may be observed, we have studied MCP-1 expression in HaCaT, SH-SY5Y, THP-1 and K562 exposed to a sinusoidal EMF at 50 Hz frequency with a flux density of 1 mT (rms).Our preliminary results showed that EMF-exposure differently modifies the expression of MCP-1 in different cell types. Thus, the MCP-1 expression needs to be better determined, with additional studies, with different parameters and times of exposure to ELF-EMF.Abbreviations: AD, Alzheimer’s disease, ELF, extremely low frequency, EMFs, electromagnetic fields, HD, Huntington disease, LF, low frequency, MCP-1, monocyte chemoattractant protein-1, PMA, phorbol-12-myristate-13-acetate, PEMF, pulsed EMF     

Effects of extremely low‐frequency magnetic field exposure on cognitive functions: results of a meta‐analysis

There is extensive literature on possible effects of extremely low‐frequency magnetic fields (ELF‐MFs) on human cognitive functions. However, due to methodological deficits (e.g., low statistical power, small sample sizes) findings have been inconsistent. In the current study we try to overcome these problems by carrying out a meta‐analysis. Literature research revealed 17 studies. Nine of these were included in the meta‐analysis because they fulfilled minimum requirements (e.g., at least single‐blind experimental study design and documentation of means and standard deviation of the dependent variables). All of the studies used a 50 Hz magnetic field exposure. Small but significant effect sizes could be detected in two cognitive dimensions: in the hard level of visual duration discrimination, task‐exposed subjects performed better than controls; at the intermediate level however, exposed subjects performed worse. Additionally, a significant improvement of correct responses was observed in the dimension of “flexibility” under exposure. However, due to the small number of studies per performance dimensions and the resulting instability of estimates, these findings have to be treated with extreme caution. Taken together, the results of the meta‐analysis provide little evidence that ELF‐MFs have any effects on cognitive functions. Bioelectromagnetics 31:173–179, 2010. © 2009 Wiley‐Liss, Inc.     

Problems of sampling and inference in the study of fluctuating dental asymmetry

Randomly distributed or “fluctuating” dental asymmetry has been accorded evolutionary meaning and interpreted as a result of environmental stress. However, except for congenital malformation syndromes, the determinants of human crown size asymmetry are still equivocal. Both a computer simulated sampling experiment using a combined sample size of N = 3000, and the requirements of adequate statistical power show that sample sizes of several hundred are needed to detect population differences in dental asymmetry. Using the largest available sample of children with defined prenatal stresses, we are unable to find systematic increases in crown size asymmetry. Given sampling limitations and the current inability to link increased human dental asymmetry to defined prenatal stresses, we suggest that fluctuating dental asymmetry is not yet established as a useful and reliable measure of general stress in human populations.     

Analgesic effect of the electromagnetic resonant frequencies derived from the NMR spectrum of morphine

Exposure to various types of electromagnetic fields (EMFs) affects pain specificity (nociception) and pain inhibition (analgesia). Previous study of ours has shown that exposure to the resonant spectra derived from biologically active substances' NMR may induce to live targets the same effects as the substances themselves. The purpose of this study is to investigate the potential analgesic effect of the resonant EMFs derived from the NMR spectrum of morphine. Twenty five Wistar rats were divided into five groups: control group; intraperitoneal administration of morphine 10 mg/kg body wt; exposure of rats to resonant EMFs of morphine; exposure of rats to randomly selected non resonant EMFs; and intraperitoneal administration of naloxone and simultaneous exposure of rats to the resonant EMFs of morphine. Tail Flick and Hot Plate tests were performed for estimation of the latency time. Results showed that rats exposed to NMR spectrum of morphine induced a significant increase in latency time at time points (p < 0.05), while exposure to the non resonant random EMFs exerted no effects. Additionally, naloxone administration inhibited the analgesic effects of the NMR spectrum of morphine. Our results indicate that exposure of rats to the resonant EMFs derived from the NMR spectrum of morphine may exert on animals similar analgesic effects to morphine itself.     

Power calculation for cross-sectional stepped wedge cluster randomized trials with variable cluster sizes

Standard sample size calculation formulas for stepped wedge cluster randomized trials (SW-CRTs) assume that cluster sizes are equal. When cluster sizes vary substantially, ignoring this variation may lead to an under-powered study. We investigate the relative efficiency of a SW-CRT with varying cluster sizes to equal cluster sizes, and derive variance estimators for the intervention effect that account for this variation under a mixed effects model—a commonly used approach for analyzing data from cluster randomized trials. When cluster sizes vary, the power of a SW-CRT depends on the order in which clusters receive the intervention, which is determined through randomization. We first derive a variance formula that corresponds to any particular realization of the randomized sequence and propose efficient algorithms to identify upper and lower bounds of the power. We then obtain an “expected” power based on a first-order approximation to the variance formula, where the expectation is taken with respect to all possible randomization sequences. Finally, we provide a variance formula for more general settings where only the cluster size arithmetic mean and coefficient of variation, instead of exact cluster sizes, are known in the design stage. We evaluate our methods through simulations and illustrate that the average power of a SW-CRT decreases as the variation in cluster sizes increases, and the impact is largest when the number of clusters is small.     

Macroinvertebrates associated with submerged macrophytes: sample size and power to detect effects

When planning and conducting ecological experiments, it is important to consider how many samples are necessary to detect differences among treatments with acceptably high statistical power. An analysis of statistical power is especially important when studying epiphytic macroinvertebrate colonization of submerged plants because they exhibit large plant-to-plant variability. Despite this variability, many studies have suggested that epiphytic macroinvertebrates preferentially colonize plants based on plant architecture type (broad versus dissected leaves). In this study, we calculated the power and number of samples necessary to detect differences in epiphytic macroinvertebrate abundance (numbers and biomass) among five species and two architecture types of macrophytes in a lake in MI, U.S.A. Using power analysis, we found that we had very high power to detect the differences present between macroinvertebrate abundance by architecture type and by macrophyte species (power = 1.000 and 0.994; effect sizes = 0.872 and 0.646, respectively. However, to detect very small differences between the two architecture types and the five plant species, we determined that many more samples were necessary to achieve similar statistical power (effect size = 0.1–0.3, number of samples = 60–527 and 36–310, respectively; power = 0.9). Our results suggest that macroinvertebrate abundance does in fact vary predictably with plant architecture. Dissected-leaf plants harbored higher abundances of macroinvertebrates than broad-leaf plants (ANOVA, density p = 0.001, biomass p < 0.001). This knowledge should allow us to better design future studies of epiphytic macroinvertebrates.     

Territorial male bullfrogs (Rana catesbeiana) do not assess fighting ability based on size-related variation in acoustic signals

Some animals use communication signals to assess their opponent'ssize and fighting ability during aggressive conflicts. Malefrogs assess their opponent's size based on the fundamentalfrequency (pitch) of advertisement calls, which is negativelycorrelated with body size, an important determinant of fightingability in frogs. I conducted a field playback experiment to investigate whether territorial male bullfrogs assess the sizeof opponents based solely on size-related variation in fundamentalfrequency. I repeatedly broadcast synthetic bullfrog advertisementcalls to three groups of males. Playback stimuli simulateda large male (n = 24), a small male (n = 24), or an acousticallysize-matched male (n = 34). Neither the simulated size of theopponent, the subject's own size, nor the degree of size asymmetrybetween the subject and simulated intruder had significanteffects on the magnitude of responses during the playback testor on the rate of habituation that occurred with repeated stimulation.Post-hoc analyses of effect sizes and statistical power indicatedthat the effects in this study were quite small compared toprevious studies in other frogs. More important, power analysesindicated that this study had high power (1 - ß >0.90) to detect the magnitude of effect sizes observed in previous studies. Thus, territorial male bullfrogs do not appear to assessan opponent's fighting ability based solely on the fundamentalfrequency of acoustic signals. These results contrast starklywith theoretical predictions and previous empirical work withfrogs.     

Effect of ultraviolet B radiation and 100 Hz electromagnetic fields on proliferation and DNA synthesis of Jurkat cells

The use of ultraviolet B light (UVB) has been proven to be highly effective for treatment of various inflammatory skin diseases, but UVB phototherapy is limited by its carcinogenic side effects. It is necessary to uncover effectors that augment UVB so that similar or improved efficacy can be obtained with lower UVB doses. We found that low frequency, low intensity electromagnetic fields (EMFs) can act as such an effector and synergistically inhibit T lymphocyte proliferation. We first characterized the effects of UVB on Jurkat cells, a model for cutaneous T lymphocytes, and determined UVB's dose dependent inhibition of cell proliferation and induction of apoptosis. Cells exposed to a sublethal UVB dose retained their sensitivity to UVB, but repetitive irradiation seemed to cause accumulation of delayed DNA damage. We then exposed cells to combinations of UVB plus EMFs and found that 100 Hz, 1 mT EMFs decrease DNA synthesis of UVB-activated Jurkat cells by 34 +/- 13% compared to UVB alone. The decrease is, however, most effective when relatively high UVB doses are employed. Since EMFs alone had only a very weak inhibitory effect (10 +/- 2%), the data suggest that EMFs augment the cell killing effects of UVB in a synergistic way. These findings could provide the basis for development of new and improved clinical phototherapy protocols.     

50-Hertz electromagnetic fields induce gammaH2AX foci formation in mouse preimplantation embryos in vitro

Effects of electromagnetic fields (EMFs) on DNA damage in mammals are still controversial. In the present study, the effects of EMFs on DNA damage in preimplantation mouse embryos in vitro were investigated by using gammaH2AX foci formation, a new sensitive indicator for detecting DNA double-strand breaks (DSBs). The data obtained demonstrated that EMFs decreased the cleavage rate of preimplantation mouse embryos. This decreasing effect of EMFs was related to the DNA-damaging effect indicated by the induction of gammaH2AX foci formation in preimplantation mouse embryos. The inducing effects of EMFs on gammaH2AX foci formation could be inhibited by the treatment of noise MFs or wortmannin, a phosphatidylinositol 3-kinase (PI3K) family inhibitor. Furthermore, the data obtained also showed that EMFs could activate the DNA damage-repair mechanism by recruiting repair factor Rad50 to the damaged DNA sites to repair the corresponding DNA damage. These findings suggest that EMFs could cause DNA damage in preimplantation embryos in vitro and that the adverse effects of EMFs on development might at least partly act through DNA damage. The DNA damage induced by EMFs could be at least partly repaired by the natural activation of DNA damage-repair mechanism or prevented by the simultaneous treatment of noise magnetic fields.     

Neurophysiological effects of mobile phone electromagnetic fields on humans: a comprehensive review

In recent years a growing number of people have begun to use mobile phone technology. This phenomenon has raised questions and doubts about possible effects on users' brains. This literature review focuses on the human electrophysiological and neuro-metabolic effects of mobile phone (MP)-related electromagnetic fields (EMFs) published in the last 10 years. To this end, all relevant papers have been reported and, subsequently, a literature selection has been carried out by taking several criteria into account, such as: blind techniques, randomization or counter-balancing of conditions and subjects, detail of exposure characteristics and the statistical analyses used. As a result, only the studies meeting the selection criteria have been described, evaluated and discussed further. The main goal of this review is to provide a clear scenario of the most reliable experiments carried out over the last decade and to offer a critical point of view in their evaluation. It is concluded that MP-EMFs may influence normal physiology through changes in cortical excitability and that in future research particular care should be dedicated to both methodological and statistical control, the most relevant criteria in this research field.     

Use of High-Frequency In-Home Monitoring Data May Reduce Sample Sizes Needed in Clinical Trials

Background

Trials in Alzheimer’s disease are increasingly focusing on prevention in asymptomatic individuals. This poses a challenge in examining treatment effects since currently available approaches are often unable to detect cognitive and functional changes among asymptomatic individuals. Resultant small effect sizes require large sample sizes using biomarkers or secondary measures for randomized controlled trials (RCTs). Better assessment approaches and outcomes capable of capturing subtle changes during asymptomatic disease stages are needed.

Objective

We aimed to develop a new approach to track changes in functional outcomes by using individual-specific distributions (as opposed to group-norms) of unobtrusive continuously monitored in-home data. Our objective was to compare sample sizes required to achieve sufficient power to detect prevention trial effects in trajectories of outcomes in two scenarios: (1) annually assessed neuropsychological test scores (a conventional approach), and (2) the likelihood of having subject-specific low performance thresholds, both modeled as a function of time.

Methods

One hundred nineteen cognitively intact subjects were enrolled and followed over 3 years in the Intelligent Systems for Assessing Aging Change (ISAAC) study. Using the difference in empirically identified time slopes between those who remained cognitively intact during follow-up (normal control, NC) and those who transitioned to mild cognitive impairment (MCI), we estimated comparative sample sizes required to achieve up to 80% statistical power over a range of effect sizes for detecting reductions in the difference in time slopes between NC and MCI incidence before transition.

Results

Sample size estimates indicated approximately 2000 subjects with a follow-up duration of 4 years would be needed to achieve a 30% effect size when the outcome is an annually assessed memory test score. When the outcome is likelihood of low walking speed defined using the individual-specific distributions of walking speed collected at baseline, 262 subjects are required. Similarly for computer use, 26 subjects are required.

Conclusions

Individual-specific thresholds of low functional performance based on high-frequency in-home monitoring data distinguish trajectories of MCI from NC and could substantially reduce sample sizes needed in dementia prevention RCTs.     

Multiple comparisons distortions of parameter estimates

In experiments involving many variables, investigators typically use multiple comparisons procedures to determine differences that are unlikely to be the result of chance. However, investigators rarely consider how the magnitude of the greatest observed effect sizes may have been subject to bias resulting from multiple testing. These questions of bias become important to the extent investigators focus on the magnitude of the observed effects. As an example, such bias can lead to problems in attempting to validate results, if a biased effect size is used to power a follow-up study. An associated important consequence is that confidence intervals constructed using standard distributions may be badly biased. A bootstrap approach is used to estimate and adjust for the bias in the effect sizes of those variables showing strongest differences. This bias is not always present; some principles showing what factors may lead to greater bias are given and a proof of the convergence of the bootstrap distribution is provided.