Case studies in reproducibility

Reproducible research is a concept of providing access to data and software along with published scientific findings. By means of some case studies from different disciplines, we will illustrate reasons why readers should be given the possibility to look at the data and software independently from the authors of the original publication. We report results of a survey comprising 100 papers recently published in Bioinformatics. The main finding is that authors of this journal share a culture of making data available. However, the number of papers where source code for simulation studies or analyzes is available is still rather limited.     

Assessment of the reproducibility of nocturnal growth hormone secretion

The reproducibility of measurements of nocturnal spontaneous secretion of growth hormone (GH) was assessed. The study population consisted of 15 short children with a variety of pathological conditions. Blood samples were taken every 20 min from each subject during sleep on two consecutive nights. The analysis of variance of matched series indicated a global reproducibility, but also demonstrated significant individual variabilities (with-in-subject effect) of both peak amplitude (p = 0.05) and mean concentration (p = 0.02). We did not find any link between the variation of the GH parameters and the variations of the clinical sleep score in 11 patients.     

Master of laboratory animal science program: teaching the science of laboratory animal science

The administrators of the only program of its kind in the US explain the demand for a Master's degree in Laboratory Animal Science, the value of the degree, and plans to expand the program to make it available for distance learning.     

Peptones and mycological reproducibility.

The pH reactions, ultraviolet spectra and phosphorus content of solutions of a variety of commercially available peptones all indicated, predictably, considerable differences in the chemical composition of the peptones. The effects of these differences on the outcome of experiments with Candida albicans grown in different peptone media was investigated. The fungus produced germ tubes equally effectively on all such media provided that the inoculum was kept to 10(6) blastospores/ml or less. However, expression of inducible enzyme activities in C. albicans varied extensively from peptone to peptone; there was, for example, an inverse relationship between the inorganic phosphorus content of peptones and the amount of acid phosphomonoesterase detectable in intact blastospores. The results indicated that use of different peptones in "Sabouraud's" media by different laboratories may account for some, but not all, published instances of irreproducibility of experiments with C. albicans.     

Role of environmental factors on the reproducibility of Lemna test

Lemna minor is a species easy to collect and culture in laboratory, and can give rapid test results. However, in order to standardise toxicity tests using Lemna minor as test organism, it is important to find out what natural variability different populations might have. Five Lemna populations were used for comparison. It contained two standard cultures and three populations collected in natural habitats. Potassium dichromate was applied as test material. Lemna populations cultured under the same conditions showed different T(D) and LC50 values. There is an inverse relation between the sensitivity and T(D) of the strains. It is supposed that growth rate and sensitivity of Lemna populations depend on environmental factors characterising the habitat in which the given popluation originally lives.     

Examination of reproducibility in microbiological degradation experiments

Experimental data indicate that certain microbiological degradation experiments have a limited reproducibility. Nine identical batch experiments were carried out on 3 different days to examine reproducibility. A pure culture, isolated from soil, grew with toluene as the only carbon and energy source. Toluene was degraded under aerobic conditions at a constant temperature of 28 °C. The experiments were modelled by a Monod model – extended to meet the air/liquid system, and the parameter values were estimated using a statistical nonlinear estimation procedure. Model reduction analysis resulted in a simpler model without the biomass decay term. In order to test for model reduction and reproducibility of parameter estimates, a likelihood ratio test was employed. The limited reproducibility for these experiments implied that all 9 batch experiments could not be described by the same set of parameter values. However, experiments carried out the same day (within the same run) were more uniform than experiments carried out on different days (between runs), and a common set of parameter estimates could be accepted for experiments within runs, but not for experiments from different runs. The limited reproducibility may be caused by variability in the preculture, or more precisely, variations in the physiological state of the bacteria in the precultures just before used as inoculum.