World Health Organization Estimates of the Relative Contributions of Food to the Burden of Disease Due to Selected Foodborne Hazards: A Structured Expert Elicitation

Background

The Foodborne Disease Burden Epidemiology Reference Group (FERG) was established in 2007 by the World Health Organization (WHO) to estimate the global burden of foodborne diseases (FBDs). This estimation is complicated because most of the hazards causing FBD are not transmitted solely by food; most have several potential exposure routes consisting of transmission from animals, by humans, and via environmental routes including water. This paper describes an expert elicitation study conducted by the FERG Source Attribution Task Force to estimate the relative contribution of food to the global burden of diseases commonly transmitted through the consumption of food.

Methods and Findings

We applied structured expert judgment using Cooke’s Classical Model to obtain estimates for 14 subregions for the relative contributions of different transmission pathways for eleven diarrheal diseases, seven other infectious diseases and one chemical (lead). Experts were identified through international networks followed by social network sampling. Final selection of experts was based on their experience including international working experience. Enrolled experts were scored on their ability to judge uncertainty accurately and informatively using a series of subject-matter specific ‘seed’ questions whose answers are unknown to the experts at the time they are interviewed. Trained facilitators elicited the 5^th, and 50^th and 95^th percentile responses to seed questions through telephone interviews. Cooke’s Classical Model uses responses to the seed questions to weigh and aggregate expert responses. After this interview, the experts were asked to provide 5^th, 50^th, and 95^th percentile estimates for the ‘target’ questions regarding disease transmission routes. A total of 72 experts were enrolled in the study. Ten panels were global, meaning that the experts should provide estimates for all 14 subregions, whereas the nine panels were subregional, with experts providing estimates for one or more subregions, depending on their experience in the region. The size of the 19 hazard-specific panels ranged from 6 to 15 persons with several experts serving on more than one panel. Pathogens with animal reservoirs (e.g. non-typhoidal Salmonella spp. and Toxoplasma gondii) were in general assessed by the experts to have a higher proportion of illnesses attributable to food than pathogens with mainly a human reservoir, where human-to-human transmission (e.g. Shigella spp. and Norovirus) or waterborne transmission (e.g. Salmonella Typhi and Vibrio cholerae) were judged to dominate. For many pathogens, the foodborne route was assessed relatively more important in developed subregions than in developing subregions. The main exposure routes for lead varied across subregions, with the foodborne route being assessed most important only in two subregions of the European region.

Conclusions

For the first time, we present worldwide estimates of the proportion of specific diseases attributable to food and other major transmission routes. These findings are essential for global burden of FBD estimates. While gaps exist, we believe the estimates presented here are the best current source of guidance to support decision makers when allocating resources for control and intervention, and for future research initiatives.     

Layman versus Professional Musician: Who Makes the Better Judge?

The increasing number of casting shows and talent contests in the media over the past years suggests a public interest in rating the quality of vocal performances. In many of these formats, laymen alongside music experts act as judges. Whereas experts'' judgments are considered objective and reliable when it comes to evaluating singing voice, little is known about laymen’s ability to evaluate peers. On the one hand, layman listeners–who by definition did not have any formal training or regular musical practice–are known to have internalized the musical rules on which singing accuracy is based. On the other hand, layman listeners’ judgment of their own vocal skills is highly inaccurate. Also, when compared with that of music experts, their level of competence in pitch perception has proven limited. The present study investigates laypersons'' ability to objectively evaluate melodies performed by untrained singers. For this purpose, laymen listeners were asked to judge sung melodies. The results were compared with those of music experts who had performed the same task in a previous study. Interestingly, the findings show a high objectivity and reliability in layman listeners. Whereas both the laymen''s and experts'' definition of pitch accuracy overlap, differences regarding the musical criteria employed in the rating task were evident. The findings suggest that the effect of expertise is circumscribed and limited and supports the view that laypersons make trustworthy judges when evaluating the pitch accuracy of untrained singers.     

Scientists versus Regulators: Precaution,Novelty & Regulatory Oversight as Predictors of Perceived Risks of Engineered Nanomaterials

Engineered nanoscale materials (ENMs) present a difficult challenge for risk assessors and regulators. Continuing uncertainty about the potential risks of ENMs means that expert opinion will play an important role in the design of policies to minimize harmful implications while supporting innovation. This research aims to shed light on the views of ‘nano experts’ to understand which nanomaterials or applications are regarded as more risky than others, to characterize the differences in risk perceptions between expert groups, and to evaluate the factors that drive these perceptions. Our analysis draws from a web-survey (N = 404) of three groups of US and Canadian experts: nano-scientists and engineers, nano-environmental health and safety scientists, and regulatory scientists and decision-makers. Significant differences in risk perceptions were found across expert groups; differences found to be driven by underlying attitudes and perceptions characteristic of each group. Nano-scientists and engineers at the upstream end of the nanomaterial life cycle perceived the lowest levels of risk, while those who are responsible for assessing and regulating risks at the downstream end perceived the greatest risk. Perceived novelty of nanomaterial risks, differing preferences for regulation (i.e. the use of precaution versus voluntary or market-based approaches), and perceptions of the risk of technologies in general predicted variation in experts'' judgments of nanotechnology risks. Our findings underscore the importance of involving a diverse selection of experts, particularly those with expertise at different stages along the nanomaterial lifecycle, during policy development.     

Evaluating perceptions of risk in mosquito experts and identifying undocumented pathways for the introduction of invasive mosquito species into Europe

In several reported cases of the entry of invasive mosquito species (IMSs) into Europe, the introduction was associated with a specific pathway of introduction or dispersal. The identification of potential pathways for the introduction of IMSs and evaluations of the importance of the different pathways are key to designing proper surveillance strategies to promptly detect and control introductions in non‐infested areas. The main goals of the present study were to identify other, previously undocumented, pathways of introduction into Europe, and to identify mosquito experts' perceptions regarding control measures against IMS introductions via different documented pathways. At the European Mosquito Control Association (EMCA) conference in Montenegro in March 2017, a questionnaire was distributed among meeting participants to collect expert data. Results showed that ground transportation (by cars, trucks, etc.), passive natural dispersal and the shipping of used tyres are perceived as the most likely pathways. Introduction via aircraft did not appear to be well known and was not perceived as probable. This study shows that there were no pathways unknown to European experts that could lead to cryptic introductions into the experts' countries. Furthermore, the findings demonstrated that the perceived efficacy of surveillance and control is key to overcoming the constraints experienced and to supporting the implementation of actions against introductions.     

Rank and Order: Evaluating the Performance of SNPs for Individual Assignment in a Non-Model Organism

Single nucleotide polymorphisms (SNPs) are valuable tools for ecological and evolutionary studies. In non-model species, the use of SNPs has been limited by the number of markers available. However, new technologies and decreasing technology costs have facilitated the discovery of a constantly increasing number of SNPs. With hundreds or thousands of SNPs potentially available, there is interest in comparing and developing methods for evaluating SNPs to create panels of high-throughput assays that are customized for performance, research questions, and resources. Here we use five different methods to rank 43 new SNPs and 71 previously published SNPs for sockeye salmon: F_ST, informativeness (I_n), average contribution to principal components (LC), and the locus-ranking programs BELS and WHICHLOCI. We then tested the performance of these different ranking methods by creating 48- and 96-SNP panels of the top-ranked loci for each method and used empirical and simulated data to obtain the probability of assigning individuals to the correct population using each panel. All 96-SNP panels performed similarly and better than the 48-SNP panels except for the 96-SNP BELS panel. Among the 48-SNP panels, panels created from F_ST, I_n, and LC ranks performed better than panels formed using the top-ranked loci from the programs BELS and WHICHLOCI. The application of ranking methods to optimize panel performance will become more important as more high-throughput assays become available.     

Measuring Laypeople’s Trust in Experts in a Digital Age: The Muenster Epistemic Trustworthiness Inventory (METI)

Given their lack of background knowledge, laypeople require expert help when dealing with scientific information. To decide whose help is dependable, laypeople must judge an expert’s epistemic trustworthiness in terms of competence, adherence to scientific standards, and good intentions. Online, this may be difficult due to the often limited and sometimes unreliable source information available. To measure laypeople’s evaluations of experts (encountered online), we constructed an inventory to assess epistemic trustworthiness on the dimensions expertise, integrity, and benevolence. Exploratory (n = 237) and confirmatory factor analyses (n = 345) showed that the Muenster Epistemic Trustworthiness Inventory (METI) is composed of these three factors. A subsequent experimental study (n = 137) showed that all three dimensions of the METI are sensitive to variation in source characteristics. We propose using this inventory to measure assignments of epistemic trustworthiness, that is, all judgments laypeople make when deciding whether to place epistemic trust in–and defer to–an expert in order to solve a scientific informational problem that is beyond their understanding.     

The role of human factor in maritime environment risk assessment: A practical application on Ballast Water Treatment (BWT) system in ship

The role of human factor plays a critical role in the safe and clean operation of maritime industry. Human error prediction can be beneficial to assess risk in maritime industry since shipping activities can pose potential hazards to human life and maritime ecology. The aim of this paper is to propose a risk assessment tool by considering the role of human factor. Hence, the desired safety control level in maritime transportation activities can be ascertained. In the proposed approach, a Success Likelihood Index Method (SLIM) extended with fuzzy logic is used to calculate human error probability (HEP). Severity of consequences are adopted in the proposed approach to assess risk. The quantitative risk assessment approach under fuzzy SLIM methodology will be applied to a very specific case on-board ship: Ballast Water Treatment (BWT) system. In order to improve consistency of research and minimize subjectivity of experts' judgments, the paper adopts the dominance factor which is used to adjust the impact level of experts' judgments in the aggregation stage of the methodology. The paper aims at not only highlighting the importance of human factor in maritime risk assessment but also enhancing safety control level and minimizing potential environmental impacts to marine ecology.     

Genomics Assisted Ancestry Deconvolution in Grape

The genus Vitis (the grapevine) is a group of highly diverse, diploid woody perennial vines consisting of approximately 60 species from across the northern hemisphere. It is the world’s most valuable horticultural crop with ~8 million hectares planted, most of which is processed into wine. To gain insights into the use of wild Vitis species during the past century of interspecific grape breeding and to provide a foundation for marker-assisted breeding programmes, we present a principal components analysis (PCA) based ancestry estimation method to calculate admixture proportions of hybrid grapes in the United States Department of Agriculture grape germplasm collection using genome-wide polymorphism data. We find that grape breeders have backcrossed to both the domesticated V. vinifera and wild Vitis species and that reasonably accurate genome-wide ancestry estimation can be performed on interspecific Vitis hybrids using a panel of fewer than 50 ancestry informative markers (AIMs). We compare measures of ancestry informativeness used in selecting SNP panels for two-way admixture estimation, and verify the accuracy of our method on simulated populations of admixed offspring. Our method of ancestry deconvolution provides a first step towards selection at the seed or seedling stage for desirable admixture profiles, which will facilitate marker-assisted breeding that aims to introgress traits from wild Vitis species while retaining the desirable characteristics of elite V. vinifera cultivars.     

Comparison of genotype imputation strategies using a combined reference panel for chicken population

Using whole-genome sequence (WGS) data are supposed to be optimal for genome-wide association studies and genomic predictions. However, sequencing thousands of individuals of interest is expensive. Imputation from single nucleotide polymorphisms panels to WGS data is an attractive approach to obtain highly reliable WGS data at low cost. Here, we conducted a genotype imputation study with a combined reference panel in yellow-feather dwarf broiler population. The combined reference panel was assembled by sequencing 24 key individuals of a yellow-feather dwarf broiler population (internal reference panel) and WGS data from 311 chickens in public databases (external reference panel). Three scenarios were investigated to determine how different factors affect the accuracy of imputation from 600 K array data to WGS data, including: genotype imputation with internal, external and combined reference panels; the number of internal reference individuals in the combined reference panel; and different reference sizes and selection strategies of an external reference panel. Results showed that imputation accuracy from 600 K to WGS data were 0.834±0.012, 0.920±0.007 and 0.982±0.003 for the internal, external and combined reference panels, respectively. Increasing the reference size from 50 to 250 improved the accuracy of genotype imputation from 0.848 to 0.974 for the combined reference panel and from 0.647 to 0.917 for the external reference panel. The selection strategies for the external reference panel had no impact on the accuracy of imputation using the combined reference panel. However, if only an external reference panel with reference size >50 was used, the selection strategy of minimizing the average distance to the closest leaf had the greatest imputation accuracy compared with other methods. Generally, using a combined reference panel provided greater imputation accuracy, especially for low-frequency variants. In conclusion, the optimal imputation strategy with a combined reference panel should comprehensively consider genetic diversity of the study population, availability and properties of external reference panels, sequencing and computing costs, and frequency of imputed variants. This work sheds light on how to design and execute genotype imputation with a combined external reference panel in a livestock population.     

A Mathematical Framework for Combining Decisions of Multiple Experts toward Accurate and Remote Diagnosis of Malaria Using Tele-Microscopy

We propose a methodology for digitally fusing diagnostic decisions made by multiple medical experts in order to improve accuracy of diagnosis. Toward this goal, we report an experimental study involving nine experts, where each one was given more than 8,000 digital microscopic images of individual human red blood cells and asked to identify malaria infected cells. The results of this experiment reveal that even highly trained medical experts are not always self-consistent in their diagnostic decisions and that there exists a fair level of disagreement among experts, even for binary decisions (i.e., infected vs. uninfected). To tackle this general medical diagnosis problem, we propose a probabilistic algorithm to fuse the decisions made by trained medical experts to robustly achieve higher levels of accuracy when compared to individual experts making such decisions. By modelling the decisions of experts as a three component mixture model and solving for the underlying parameters using the Expectation Maximisation algorithm, we demonstrate the efficacy of our approach which significantly improves the overall diagnostic accuracy of malaria infected cells. Additionally, we present a mathematical framework for performing ‘slide-level’ diagnosis by using individual ‘cell-level’ diagnosis data, shedding more light on the statistical rules that should govern the routine practice in examination of e.g., thin blood smear samples. This framework could be generalized for various other tele-pathology needs, and can be used by trained experts within an efficient tele-medicine platform.     

Effect of Supervised Students' Involvement on Diagnostic Accuracy in Hospitalized Medical Patients �� A Prospective Controlled Study

Background

During internships most medical students engage in history taking and physical examination during evaluation of hospitalized patients. However, the students'' ability for pattern recognition is not as developed as in medical experts and complete history taking is often not repeated by an expert, so important clues may be missed. On the other hand, students'' history taking is usually more extensive than experts'' history taking and medical students discuss their findings with a Supervisor. Thus the effect of student involvement on diagnostic accuracy is unclear. We therefore compared the diagnostic accuracy for patients in the medical emergency department with and without student involvement in the evaluation process.

Methodology/Principal Findings

Patients in the medical emergency department were assigned to evaluation by either a supervised medical student or an emergency department physician. We only included patients who were admitted to our hospital and subsequently cared for by another medical team on the ward. We compared the working diagnosis from the emergency department with the discharge diagnosis. A total of 310 patients included in the study were cared for by 41 medical students and 21 emergency department physicians. The working diagnosis was changed in 22% of the patients evaluated by physicians evaluation and in 10% of the patients evaluated by supervised medical students (p = .006). There was no difference in the expenditures for diagnostic procedures, length of stay in the emergency department or patient comorbidity complexity level.

Conclusion/Significance

Involvement of closely supervised medical students in the evaluation process of hospitalized medical patients leads to an improved diagnostic accuracy compared to evaluation by an emergency department physician alone.     

"GOLD or lower limit of normal definition? a comparison with expert-based diagnosis of chronic obstructive pulmonary disease in a prospective cohort-study"

Background

The Global initiative for chronic Obstructive Lung Disease (GOLD) defines COPD as a fixed post-bronchodilator ratio of forced expiratory volume in 1 second and forced vital capacity (FEV1/FVC) below 0.7. Age-dependent cut-off values below the lower fifth percentile (LLN) of this ratio derived from the general population have been proposed as an alternative. We wanted to assess the diagnostic accuracy and prognostic capability of the GOLD and LLN definition when compared to an expert-based diagnosis.

Methods

In a prospective cohort study, 405 patients aged ≥ 65 years with a general practitioner''s diagnosis of COPD were recruited and followed up for 4.5 (median; quartiles 3.9; 5.1) years. Prevalence rates of COPD according to GOLD and three LLN definitions and diagnostic performance measurements were calculated. The reference standard was the diagnosis of COPD of an expert panel that used all available diagnostic information, including spirometry and bodyplethysmography.

Results

Compared to the expert panel diagnosis, ''GOLD-COPD'' misclassified 69 (28%) patients, and the three LLNs misclassified 114 (46%), 96 (39%), and 98 (40%) patients, respectively. The GOLD classification led to more false positives, the LLNs to more false negative diagnoses. The main predictors beyond the FEV1/FVC ratio for an expert diagnosis of COPD were the FEV1 % predicted, and the residual volume/total lung capacity ratio (RV/TLC). Adding FEV1 and RV/TLC to GOLD or LLN improved the diagnostic accuracy, resulting in a significant reduction of up to 50% of the number of misdiagnoses. The expert diagnosis of COPD better predicts exacerbations, hospitalizations and mortality than GOLD or LLN.

Conclusions

GOLD criteria over-diagnose COPD, while LLN definitions under-diagnose COPD in elderly patients as compared to an expert panel diagnosis. Incorporating FEV1 and RV/TLC into the GOLD-COPD or LLN-based definition brings both definitions closer to expert panel diagnosis of COPD, and to daily clinical practice.     

Experts and Novices Use the Same Factors–But Differently–To Evaluate Pearl Quality

Well-trained experts in pearl grading have been thought to evaluate pearls according to their glossiness, interference color, and shape. However, the characteristics of their evaluations are not fully understood. Using pearl grading experiments, we investigate the consistency of novice (i.e., without knowledge of pearl grading) and expert participants’ pearl grading skill and then compare the novices’ grading with that of experts; furthermore, we discuss the relationship between grading, interference color, and glossiness. We found that novices’ grading was significantly less concordant with experts average grading than was experts’ grading; more than half of novices graded pearls the opposite of how experts graded those same pearls. However, while experts graded pearls more consistently than novices did, novices’ consistency was relatively high. We also found differences between the groups in regression analyses that used interference color and glossiness as explanatory variables and were conducted for each trial. Although the regression coefficient was significant in 60% of novices’ trials, there were fewer significant trials for the experts (20%). This indicates that novices can also make use of these two factors, but that their usage is simpler than that of the experts. These results suggest that experts and novices share some values about pearls but that the evaluation method is elaborated for experts.     

Short-term Forecasting of the Prevalence of Trachoma: Expert Opinion,Statistical Regression,versus Transmission Models

BackgroundTrachoma programs rely on guidelines made in large part using expert opinion of what will happen with and without intervention. Large community-randomized trials offer an opportunity to actually compare forecasting methods in a masked fashion.MethodsThe Program for the Rapid Elimination of Trachoma trials estimated longitudinal prevalence of ocular chlamydial infection from 24 communities treated annually with mass azithromycin. Given antibiotic coverage and biannual assessments from baseline through 30 months, forecasts of the prevalence of infection in each of the 24 communities at 36 months were made by three methods: the sum of 15 experts’ opinion, statistical regression of the square-root-transformed prevalence, and a stochastic hidden Markov model of infection transmission (Susceptible-Infectious-Susceptible, or SIS model). All forecasters were masked to the 36-month results and to the other forecasts. Forecasts of the 24 communities were scored by the likelihood of the observed results and compared using Wilcoxon’s signed-rank statistic.FindingsRegression and SIS hidden Markov models had significantly better likelihood than community expert opinion (p = 0.004 and p = 0.01, respectively). All forecasts scored better when perturbed to decrease Fisher’s information. Each individual expert’s forecast was poorer than the sum of experts.InterpretationRegression and SIS models performed significantly better than expert opinion, although all forecasts were overly confident. Further model refinements may score better, although would need to be tested and compared in new masked studies. Construction of guidelines that rely on forecasting future prevalence could consider use of mathematical and statistical models.     

Reduced SNP Panels for Genetic Identification and Introgression Analysis in the Dark Honey Bee (Apis mellifera mellifera)

Beekeeping activities, especially queen trading, have shaped the distribution of honey bee (Apis mellifera) subspecies in Europe, and have resulted in extensive introductions of two eastern European C-lineage subspecies (A. m. ligustica and A. m. carnica) into the native range of the M-lineage A. m. mellifera subspecies in Western Europe. As a consequence, replacement and gene flow between native and commercial populations have occurred at varying levels across western European populations. Genetic identification and introgression analysis using molecular markers is an important tool for management and conservation of honey bee subspecies. Previous studies have monitored introgression by using microsatellite, PCR-RFLP markers and most recently, high density assays using single nucleotide polymorphism (SNP) markers. While the latter are almost prohibitively expensive, the information gained to date can be exploited to create a reduced panel containing the most ancestry-informative markers (AIMs) for those purposes with very little loss of information. The objective of this study was to design reduced panels of AIMs to verify the origin of A. m. mellifera individuals and to provide accurate estimates of the level of C-lineage introgression into their genome. The discriminant power of the SNPs using a variety of metrics and approaches including the Weir & Cockerham’s F_ST, an F_ST-based outlier test, Delta, informativeness (I_n), and PCA was evaluated. This study shows that reduced AIMs panels assign individuals to the correct origin and calculates the admixture level with a high degree of accuracy. These panels provide an essential tool in Europe for genetic stock identification and estimation of admixture levels which can assist management strategies and monitor honey bee conservation programs.     

The Willingness of Children to Lie and the Assessment of Credibility in an Ecologically Relevant Laboratory Setting

A series of 4 experiments were conducted to (a) determine the willingness of children to lie in a realistic setting, (b) compare judgments of credibility from both lay evaluations and Criterion-Based Content Analysis (CBCA), and (c) examine the effects of expert testimony regarding Statement Validity Assessment on mock jurors who were asked to make evaluations of the children's statements. In Experiment 1, 81% of children who witnessed a research assistant steal a textbook made accusations against the thief (truthful), 69% of children who did not witness the theft accused the research assistant of the theft following prompting by significant others, and 56% of the children who witnessed a significant other steal the textbook incorrectly accused the research assistant following a request from their significant other. Using the statements obtained from the children in Experiment 1, Experiments 2 and 3 found that classification accuracy of lay evaluators was significantly poorer than expert application of CBCA, which resulted in 89% classification accuracy. Finally, brief exposure to CBCA expert testimony appeared to have no benefits on mock jurors' assessments of credibility of the children's statements in Experiment 4. Implications for North American legal systems are discussed.     

Contesting expertise: anthropologists at the Special Court for Sierra Leone

This article examines how social anthropologists' expertise was employed in the international war crimes trials heard at the Special Court for Sierra Leone. It tracks how the anthropologists challenged the prosecution experts by raising concerns about their methodology and advancing a fundamental critique of abstract legal categories. The discussion between the experts centred on two contested issues: the character of the armed groups and the phenomenon of forced marriages during the civil war in Sierra Leone. The analysis of the experts' testimonies shows that the anthropologists were engaged in an epistemological contest with the prosecution experts. The article aims at understanding why the anthropologists' arguments had less resonance with the judges than the reports submitted by the prosecution experts. Beyond the courtroom, it also speaks to the general debate about expertise and ways to study it.