Fundamental concepts in statistics: elucidation and illustration

Fundamentalconcepts in statistics form the cornerstone of scientific inquiry. Ifwe fail to understand fully these fundamental concepts, then thescientific conclusions we reach are more likely to be wrong. This ismore than supposition: for 60 years, statisticians have warned that thescientific literature harbors misunderstandings about basic statisticalconcepts. Original articles published in 1996 by the AmericanPhysiological Society's journals fared no better in their handling ofbasic statistical concepts. In this review, we summarize the two mainscientific uses of statistics: hypothesis testing and estimation. Mostscientists use statistics solely for hypothesis testing; often,however, estimation is more useful. We also illustrate the concepts ofvariability and uncertainty, and we demonstrate the essentialdistinction between statistical significance and scientific importance.An understanding of concepts such as variability, uncertainty, andsignificance is necessary, but it is not sufficient; we show also thatthe numerical results of statistical analyses have limitations.

     

The Use of Statistics in Health Sciences: Situation Analysis and Perspective

Statistics plays a crucial role in research, planning and decision-making in the health sciences. Progress in technologies and continued research in computational statistics has enabled us to implement sophisticated mathematical models within software that are handled by non-statistician researchers. As a result, over the last decades, medical journals have published a host of papers that use some novel statistical method. The aim of this paper is to present a review on how the statistical methods are being applied in the construction of scientific knowledge in health sciences, as well as, to propose some improvement actions. From the early twentieth century, there has been a remarkable surge in scientific evidence alerting on the errors that many non-statistician researchers were making in applying statistical methods. Today, several studies continue showing that a large percentage of articles published in high-impact factor journals contain errors in data analysis or interpretation of results, with the ensuing repercussions on the validity and efficiency of the research conducted. Scientific community should reflect on the causes that have led to this situation, the consequences to the advancement of scientific knowledge and the solutions to this problem.     

Which statistics should tropical biologists learn?

Tropical biologists study the richest and most endangered biodiversity in the planet, and in these times of climate change and mega-extinctions, the need for efficient, good quality research is more pressing than in the past. However, the statistical component in research published by tropical authors sometimes suffers from poor quality in data collection; mediocre or bad experimental design and a rigid and outdated view of data analysis. To suggest improvements in their statistical education, we listed all the statistical tests and other quantitative analyses used in two leading tropical journals, the Revista de Biología Tropical and Biotropica, during a year. The 12 most frequent tests in the articles were: Analysis of Variance (ANOVA), Chi-Square Test, Student's T Test, Linear Regression, Pearson's Correlation Coefficient, Mann-Whitney U Test, Kruskal-Wallis Test, Shannon's Diversity Index, Tukey's Test, Cluster Analysis, Spearman's Rank Correlation Test and Principal Component Analysis. We conclude that statistical education for tropical biologists must abandon the old syllabus based on the mathematical side of statistics and concentrate on the correct selection of these and other procedures and tests, on their biological interpretation and on the use of reliable and friendly freeware. We think that their time will be better spent understanding and protecting tropical ecosystems than trying to learn the mathematical foundations of statistics: in most cases, a well designed one-semester course should be enough for their basic requirements.     

Introduction to Biostatistics—An Annotated Bibliography for Medical Researchers

Biostatistical methods have become thoroughly integrated into modern biomedical and clinical research. Nevertheless, every observer who has evaluated articles in medical journals has noted that as many as half the reported results were based on questionable statistical analysis. This situation, combined with the fact that most errors involve relatively simple statistical procedures, points to the need for researchers and practitioners to be able to personally judge the quality of the statistical analyses in what they read. Fortunately, there are several excellent papers and texts available for those interested.     

Statistical Reporting Errors and Collaboration on Statistical Analyses in Psychological Science

Statistical analysis is error prone. A best practice for researchers using statistics would therefore be to share data among co-authors, allowing double-checking of executed tasks just as co-pilots do in aviation. To document the extent to which this ‘co-piloting’ currently occurs in psychology, we surveyed the authors of 697 articles published in six top psychology journals and asked them whether they had collaborated on four aspects of analyzing data and reporting results, and whether the described data had been shared between the authors. We acquired responses for 49.6% of the articles and found that co-piloting on statistical analysis and reporting results is quite uncommon among psychologists, while data sharing among co-authors seems reasonably but not completely standard. We then used an automated procedure to study the prevalence of statistical reporting errors in the articles in our sample and examined the relationship between reporting errors and co-piloting. Overall, 63% of the articles contained at least one p-value that was inconsistent with the reported test statistic and the accompanying degrees of freedom, and 20% of the articles contained at least one p-value that was inconsistent to such a degree that it may have affected decisions about statistical significance. Overall, the probability that a given p-value was inconsistent was over 10%. Co-piloting was not found to be associated with reporting errors.     

Impact of Article Language in Multi-Language Medical Journals - a Bibliometric Analysis of Self-Citations and Impact Factor

Background

In times of globalization there is an increasing use of English in the medical literature. The aim of this study was to analyze the influence of English-language articles in multi-language medical journals on their international recognition – as measured by a lower rate of self-citations and higher impact factor (IF).

Methods and Findings

We analyzed publications in multi-language journals in 2008 and 2009 using the Web of Science (WoS) of Thomson Reuters (former Institute of Scientific Information) and PubMed as sources of information. The proportion of English-language articles during the period was compared with both the share of self-citations in the year 2010 and the IF with and without self-citations. Multivariable linear regression analysis was performed to analyze these factors as well as the influence of the journals‘ countries of origin and of the other language(s) used in publications besides English.We identified 168 multi-language journals that were listed in WoS as well as in PubMed and met our criteria. We found a significant positive correlation of the share of English articles in 2008 and 2009 with the IF calculated without self-citations (Pearson r=0.56, p = <0.0001), a correlation with the overall IF (Pearson r = 0.47, p = <0.0001) and with the cites to years of IF calculation (Pearson r = 0.34, p = <0.0001), and a weak negative correlation with the share of self-citations (Pearson r = -0.2, p = 0.009). The IF without self-citations also correlated with the journal‘s country of origin – North American journals had a higher IF compared to Middle and South American or European journals.

Conclusion

Our findings suggest that a larger share of English articles in multi-language medical journals is associated with greater international recognition. Fewer self-citations were found in multi-language journals with a greater share of original articles in English.     

Growth and development of biofeedback: A bibliographic analysis

A computerized search was performed, and a bibliography was prepared on the subject of biofeedback covering the years from 1964 to 1985. Growth curves were produced for various publication media. The search produced references to 2,431 journal articles, 102 books, 79 popular magazine articles, and 551 doctoral dissertations. The journal articles were sorted according to the country of publication, language, and primary topic of the journal. Citations were found from 35 countries, written in 18 languages. All the media studied showed a period of rapid growth during the early to middle 1970s, but there was a tendency for leveling off or slight decline during the early 1980s. Publication of articles in medical journals has shown the greatest growth, and more articles are published yearly in medical journals than in journals of any other discipline or all specialty journals combined. Publication in psychological journals has shown a decline since 1977. Dental, nursing, and educational journals have shown a low rate of publication of biofeedback articles, indicating little or no growth.We thank Ken Hartman for his assistance in performing this study.     

Cognition of and Demand for Education and Teaching in Medical Statistics in China: A Systematic Review and Meta-Analysis

BackgroundAlthough a substantial number of studies focus on the teaching and application of medical statistics in China, few studies comprehensively evaluate the recognition of and demand for medical statistics. In addition, the results of these various studies differ and are insufficiently comprehensive and systematic.ObjectivesThis investigation aimed to evaluate the general cognition of and demand for medical statistics by undergraduates, graduates, and medical staff in China.MethodsWe performed a comprehensive database search related to the cognition of and demand for medical statistics from January 2007 to July 2014 and conducted a meta-analysis of non-controlled studies with sub-group analysis for undergraduates, graduates, and medical staff.ResultsThere are substantial differences with respect to the cognition of theory in medical statistics among undergraduates (73.5%), graduates (60.7%), and medical staff (39.6%). The demand for theory in medical statistics is high among graduates (94.6%), undergraduates (86.1%), and medical staff (88.3%). Regarding specific statistical methods, the cognition of basic statistical methods is higher than of advanced statistical methods. The demand for certain advanced statistical methods, including (but not limited to) multiple analysis of variance (ANOVA), multiple linear regression, and logistic regression, is higher than that for basic statistical methods. The use rates of the Statistical Package for the Social Sciences (SPSS) software and statistical analysis software (SAS) are only 55% and 15%, respectively.ConclusionThe overall statistical competence of undergraduates, graduates, and medical staff is insufficient, and their ability to practically apply their statistical knowledge is limited, which constitutes an unsatisfactory state of affairs for medical statistics education. Because the demand for skills in this area is increasing, the need to reform medical statistics education in China has become urgent.     

Negative changes in the scientific publication process in ecology: potential causes and consequences

1. Increasingly viewed to have societal impact and value, science is affected by complex changes such as globalisation and the increasing dominance of commercial interest. As a result, technical advancements, financial concerns, institutional prestige and journal proliferation have created challenges for ecological and other scientific journals and affected the perception of both researchers and the public about the science that they publish. 2. Journals are now used for more than dissemination of scientific research. Institutions use journal rankings for a variety of purposes and often require a pre‐established number of articles in hiring and budgetary decisions. Consequently, journal impact factors have achieved greater importance, and the splitting of articles into smaller parcels of information (‘salami‐slicing’) to increase numbers of publications has become more frequent. 3. Journals may prescribe upper limits to article length, even though the average length of articles for several ecological journals examined has increased over time. There are clear signs, however, that journals without length limits for articles will become rarer. In contrast to ecological journals, taxonomic journals are not following this trend. 4. Two case histories demonstrate how splitting longer ecological articles into a series of shorter ones results in both redundancy of information and actually increases the journal space used overall. Furthermore, with current rejection rates of ecological journals (often ～80%), many thin salami‐sliced articles jam the peer‐review system much longer (through resubmission after rejection) than unsliced articles previously did (e.g. when rejection rates were ～50%). In our experience, the increased pressure to publish many articles in ‘high‐impact’ journals also may decrease the attractiveness of a future scientific career in ecology to young people. 5. ‘Gatekeeping’ of journal quality has shifted from editors to reviewers, and several recent trends are apparent including: bias about appropriate statistical methods; reviewers being more rigid overall; non‐native English writers being criticised for poor communications skills; and favourable reviews being signed more often than unfavourable ones. In terms of production, outsourcing of copy editing has increased the final error rate of published material. 6. We supplemented our perceptions with those of older colleagues (～100 experienced ecologists) that responded to an informal survey on this topic (response rate: 81%). In the opinion of almost 90% of our respondents, the overall review process has changed and for 20% among them the professional quality of reviews has declined. 7. We, and many older colleagues, are convinced there have been some negative changes in the scientific publication process. If younger colleagues share this concern, we can collectively counter this deteriorating situation, because we are the key to the publishing and evaluation process.     

Misprescription and misuse of one‐tailed tests

One‐tailed statistical tests are often used in ecology, animal behaviour and in most other fields in the biological and social sciences. Here we review the frequency of their use in the 1989 and 2005 volumes of two journals (Animal Behaviour and Oecologia), their advantages and disadvantages, the extensive erroneous advice on them in both older and modern statistics texts and their utility in certain narrow areas of applied research. Of those articles with data sets susceptible to one‐tailed tests, at least 24% in Animal Behaviour and at least 13% in Oecologia used one‐tailed tests at least once. They were used 35% more frequently with nonparametric methods than with parametric ones and about twice as often in 1989 as in 2005. Debate in the psychological literature of the 1950s established the logical criterion that one‐tailed tests should be restricted to situations where there is interest only in results in one direction. ‘Interest’ should be defined; however, in terms of collective or societal interest and not by the individual investigator. By this ‘collective interest’ criterion, all uses of one‐tailed tests in the journals surveyed seem invalid. In his book Nonparametric Statistics, S. Siegel unrelentingly suggested the use of one‐tailed tests whenever the investigator predicts the direction of a result. That work has been a major proximate source of confusion on this issue, but so are most recent statistics textbooks. The utility of one‐tailed tests in research aimed at obtaining regulatory approval of new drugs and new pesticides is briefly described, to exemplify the narrow range of research situations where such tests can be appropriate. These situations are characterized by null hypotheses stating that the difference or effect size does not exceed, or is at least as great as, some ‘amount of practical interest’. One‐tailed tests rarely should be used for basic or applied research in ecology, animal behaviour or any other science.     

Educating the future generation of researchers: A cross-disciplinary survey of trends in analysis methods

Methods for data analysis in the biomedical, life, and social (BLS) sciences are developing at a rapid pace. At the same time, there is increasing concern that education in quantitative methods is failing to adequately prepare students for contemporary research. These trends have led to calls for educational reform to undergraduate and graduate quantitative research method curricula. We argue that such reform should be based on data-driven insights into within- and cross-disciplinary use of analytic methods. Our survey of peer-reviewed literature analyzed approximately 1.3 million openly available research articles to monitor the cross-disciplinary mentions of analytic methods in the past decade. We applied data-driven text mining analyses to the “Methods” and “Results” sections of a large subset of this corpus to identify trends in analytic method mentions shared across disciplines, as well as those unique to each discipline. We found that the t test, analysis of variance (ANOVA), linear regression, chi-squared test, and other classical statistical methods have been and remain the most mentioned analytic methods in biomedical, life science, and social science research articles. However, mentions of these methods have declined as a percentage of the published literature between 2009 and 2020. On the other hand, multivariate statistical and machine learning approaches, such as artificial neural networks (ANNs), have seen a significant increase in the total share of scientific publications. We also found unique groupings of analytic methods associated with each BLS science discipline, such as the use of structural equation modeling (SEM) in psychology, survival models in oncology, and manifold learning in ecology. We discuss the implications of these findings for education in statistics and research methods, as well as within- and cross-disciplinary collaboration.

A quantitative survey of >1 million published research articles reveals that while classical statistical methods remain in widespread use, multivariate statistical and machine-learning approaches have seen a significant increase; statistics curricula should be revised to take full advantage of these new analytical tools.     

Integrative Approach to Quality Assessment of Medical Journals Using Impact Factor,Eigenfactor, and Article Influence Scores

Background

Impact factor (IF) is a commonly used surrogate for assessing the scientific quality of journals and articles. There is growing discontent in the medical community with the use of this quality assessment tool because of its many inherent limitations. To help address such concerns, Eigenfactor (ES) and Article Influence scores (AIS) have been devised to assess scientific impact of journals. The principal aim was to compare the temporal trends in IF, ES, and AIS on the rank order of leading medical journals over time.

Methods

The 2001 to 2008 IF, ES, AIS, and number of citable items (CI) of 35 leading medical journals were collected from the Institute of Scientific Information (ISI) and the http://www.eigenfactor.org databases. The journals were ranked based on the published 2008 ES, AIS, and IF scores. Temporal score trends and variations were analyzed.

Results

In general, the AIS and IF values provided similar rank orders. Using ES values resulted in large changes in the rank orders with higher ranking being assigned to journals that publish a large volume of articles. Since 2001, the IF and AIS of most journals increased significantly; however the ES increased in only 51% of the journals in the analysis. Conversely, 26% of journals experienced a downward trend in their ES, while the rest experienced no significant changes (23%). This discordance between temporal trends in IF and ES was largely driven by temporal changes in the number of CI published by the journals.

Conclusion

The rank order of medical journals changes depending on whether IF, AIS or ES is used. All of these metrics are sensitive to the number of citable items published by journals. Consumers should thus consider all of these metrics rather than just IF alone in assessing the influence and importance of medical journals in their respective disciplines.     

Growth and development of biofeedback: a bibliographic analysis

A computerized search was performed, and a bibliography was prepared on the subject of biofeedback covering the years from 1964 to 1985. Growth curves were produced for various publication media. The search produced references to 2,431 journal articles, 102 books, 79 popular magazine articles, and 551 doctoral dissertations. The journal articles were sorted according to the country of publication, language, and primary topic of the journal. Citations were found from 35 countries, written in 18 languages. All the media studied showed a period of rapid growth during the early to middle 1970s, but there was a tendency for leveling off or slight decline during the early 1980s. Publication of articles in medical journals has shown the greatest growth, and more articles are published yearly in medical journals than in journals of any other discipline or all specialty journals combined. Publication in psychological journals has shown a decline since 1977. Dental, nursing, and educational journals have shown a low rate of publication of biofeedback articles, indicating little or no growth.     

Outlier Removal and the Relation with Reporting Errors and Quality of Psychological Research

Background

The removal of outliers to acquire a significant result is a questionable research practice that appears to be commonly used in psychology. In this study, we investigated whether the removal of outliers in psychology papers is related to weaker evidence (against the null hypothesis of no effect), a higher prevalence of reporting errors, and smaller sample sizes in these papers compared to papers in the same journals that did not report the exclusion of outliers from the analyses.

Methods and Findings

We retrieved a total of 2667 statistical results of null hypothesis significance tests from 153 articles in main psychology journals, and compared results from articles in which outliers were removed (N = 92) with results from articles that reported no exclusion of outliers (N = 61). We preregistered our hypotheses and methods and analyzed the data at the level of articles. Results show no significant difference between the two types of articles in median p value, sample sizes, or prevalence of all reporting errors, large reporting errors, and reporting errors that concerned the statistical significance. However, we did find a discrepancy between the reported degrees of freedom of t tests and the reported sample size in 41% of articles that did not report removal of any data values. This suggests common failure to report data exclusions (or missingness) in psychological articles.

Conclusions

We failed to find that the removal of outliers from the analysis in psychological articles was related to weaker evidence (against the null hypothesis of no effect), sample size, or the prevalence of errors. However, our control sample might be contaminated due to nondisclosure of excluded values in articles that did not report exclusion of outliers. Results therefore highlight the importance of more transparent reporting of statistical analyses.     

Research Design and Statistical Methods in Indian Medical Journals: A Retrospective Survey

Good quality medical research generally requires not only an expertise in the chosen medical field of interest but also a sound knowledge of statistical methodology. The number of medical research articles which have been published in Indian medical journals has increased quite substantially in the past decade. The aim of this study was to collate all evidence on study design quality and statistical analyses used in selected leading Indian medical journals. Ten (10) leading Indian medical journals were selected based on impact factors and all original research articles published in 2003 (N = 588) and 2013 (N = 774) were categorized and reviewed. A validated checklist on study design, statistical analyses, results presentation, and interpretation was used for review and evaluation of the articles. Main outcomes considered in the present study were – study design types and their frequencies, error/defects proportion in study design, statistical analyses, and implementation of CONSORT checklist in RCT (randomized clinical trials). From 2003 to 2013: The proportion of erroneous statistical analyses did not decrease (χ²=0.592, Φ=0.027, p=0.4418), 25% (80/320) in 2003 compared to 22.6% (111/490) in 2013. Compared with 2003, significant improvement was seen in 2013; the proportion of papers using statistical tests increased significantly (χ²=26.96, Φ=0.16, p<0.0001) from 42.5% (250/588) to 56.7 % (439/774). The overall proportion of errors in study design decreased significantly (χ²=16.783, Φ=0.12 p<0.0001), 41.3% (243/588) compared to 30.6% (237/774). In 2013, randomized clinical trials designs has remained very low (7.3%, 43/588) with majority showing some errors (41 papers, 95.3%). Majority of the published studies were retrospective in nature both in 2003 [79.1% (465/588)] and in 2013 [78.2% (605/774)]. Major decreases in error proportions were observed in both results presentation (χ²=24.477, Φ=0.17, p<0.0001), 82.2% (263/320) compared to 66.3% (325/490) and interpretation (χ²=25.616, Φ=0.173, p<0.0001), 32.5% (104/320) compared to 17.1% (84/490), though some serious ones were still present. Indian medical research seems to have made no major progress regarding using correct statistical analyses, but error/defects in study designs have decreased significantly. Randomized clinical trials are quite rarely published and have high proportion of methodological problems.     

How important is the statistical approach for analyzing categorical data? A critique using artificial nests

Ecologists rely heavily upon statistics to make inferences concerning ecological phenomena and to make management recommendations. It is therefore important to use statistical tests that are most appropriate for a given data-set. However, inappropriate statistical tests are often used in the analysis of studies with categorical data (i.e. count data or binary data). Since many types of statistical tests have been used in artificial nests studies, a review and comparison of these tests provides an opportunity to demonstrate the importance of choosing the most appropriate statistical approach for conceptual reasons as well as type I and type II errors.
Artificial nests have routinely been used to study the influences of habitat fragmentation, and habitat edges on nest predation. I review the variety of statistical tests used to analyze artificial nest data within the framework of the generalized linear model and argue that logistic regression is the most appropriate and flexible statistical test for analyzing binary data-sets. Using artificial nest data from my own studies and an independent data set from the medical literature as examples, I tested equivalent data using a variety of statistical methods. I then compared the p-values and the statistical power of these tests. Results vary greatly among statistical methods. Methods inappropriate for analyzing binary data often fail to yield significant results even when differences between study groups appear large, while logistic regression finds these differences statistically significant. Statistical power is is 2–3 times higher for logistic regression than for other tests. I recommend that logistic regression be used to analyze artificial nest data and other data-sets with binary data.     

Effect size, confidence interval and statistical significance: a practical guide for biologists

Null hypothesis significance testing (NHST) is the dominant statistical approach in biology, although it has many, frequently unappreciated, problems. Most importantly, NHST does not provide us with two crucial pieces of information: (1) the magnitude of an effect of interest, and (2) the precision of the estimate of the magnitude of that effect. All biologists should be ultimately interested in biological importance, which may be assessed using the magnitude of an effect, but not its statistical significance. Therefore, we advocate presentation of measures of the magnitude of effects (i.e. effect size statistics) and their confidence intervals (CIs) in all biological journals. Combined use of an effect size and its CIs enables one to assess the relationships within data more effectively than the use of p values, regardless of statistical significance. In addition, routine presentation of effect sizes will encourage researchers to view their results in the context of previous research and facilitate the incorporation of results into future meta-analysis, which has been increasingly used as the standard method of quantitative review in biology. In this article, we extensively discuss two dimensionless (and thus standardised) classes of effect size statistics: d statistics (standardised mean difference) and r statistics (correlation coefficient), because these can be calculated from almost all study designs and also because their calculations are essential for meta-analysis. However, our focus on these standardised effect size statistics does not mean unstandardised effect size statistics (e.g. mean difference and regression coefficient) are less important. We provide potential solutions for four main technical problems researchers may encounter when calculating effect size and CIs: (1) when covariates exist, (2) when bias in estimating effect size is possible, (3) when data have non-normal error structure and/or variances, and (4) when data are non-independent. Although interpretations of effect sizes are often difficult, we provide some pointers to help researchers. This paper serves both as a beginner's instruction manual and a stimulus for changing statistical practice for the better in the biological sciences.