A Delphi Technology Foresight Study: Mapping Social Construction of Scientific Evidence on Metagenomics Tests for Water Safety

Access to clean water is a grand challenge in the 21^st century. Water safety testing for pathogens currently depends on surrogate measures such as fecal indicator bacteria (e.g., E. coli). Metagenomics concerns high-throughput, culture-independent, unbiased shotgun sequencing of DNA from environmental samples that might transform water safety by detecting waterborne pathogens directly instead of their surrogates. Yet emerging innovations such as metagenomics are often fiercely contested. Innovations are subject to shaping/construction not only by technology but also social systems/values in which they are embedded, such as experts’ attitudes towards new scientific evidence. We conducted a classic three-round Delphi survey, comprised of 107 questions. A multidisciplinary expert panel (n = 24) representing the continuum of discovery scientists and policymakers evaluated the emergence of metagenomics tests. To the best of our knowledge, we report here the first Delphi foresight study of experts’ attitudes on (1) the top 10 priority evidentiary criteria for adoption of metagenomics tests for water safety, (2) the specific issues critical to governance of metagenomics innovation trajectory where there is consensus or dissensus among experts, (3) the anticipated time lapse from discovery to practice of metagenomics tests, and (4) the role and timing of public engagement in development of metagenomics tests. The ability of a test to distinguish between harmful and benign waterborne organisms, analytical/clinical sensitivity, and reproducibility were the top three evidentiary criteria for adoption of metagenomics. Experts agree that metagenomic testing will provide novel information but there is dissensus on whether metagenomics will replace the current water safety testing methods or impact the public health end points (e.g., reduction in boil water advisories). Interestingly, experts view the publics relevant in a “downstream capacity” for adoption of metagenomics rather than a co-productionist role at the “upstream” scientific design stage of metagenomics tests. In summary, these findings offer strategic foresight to govern metagenomics innovations symmetrically: by identifying areas where acceleration (e.g., consensus areas) and deceleration/reconsideration (e.g., dissensus areas) of the innovation trajectory might be warranted. Additionally, we show how scientific evidence is subject to potential social construction by experts’ value systems and the need for greater upstream public engagement on metagenomics innovations.     

International Multispecialty Consensus on How to Evaluate Ultrasound Competence: A Delphi Consensus Survey

Objectives

To achieve international consensus across multiple specialties on a generic ultrasound rating scale using a Delphi technique.

Methods

Ultrasound experts from Obstetrics-Gynaecology, Surgery, Urology, Radiology, Rheumatology, Emergency Medicine, and Gastro-Enterology representing North America, Australia, and Europe were identified. A multi-round survey was conducted to obtain consensus between these experts. Of 60 invited experts, 44 experts agreed to participate in the first Delphi round, 41 remained in the second round, and 37 completed the third Delphi round. Seven key elements of the ultrasound examination were identified from existing literature and recommendations from international ultrasound societies. All experts rated the importance of these seven elements on a five-point Likert scale in the first round and suggested potential new elements for the assessment of ultrasound skills. In the second round, the experts re-rated all elements and a third round was conducted to allow final comments. Agreement on which elements to include in the final rating scale was pre-defined as more than 80% of the experts rating an element four or five, on importance to the ultrasound examination.

Results

Two additional elements were suggested by more than 10% of the experts in the first Delphi round. Consensus was obtained to include these two new elements along with five of the original elements in the final assessment instrument: 1) Indication for the examination 2) Applied knowledge of ultrasound equipment 3) Image optimization 4) Systematic examination 5) Interpretation of images 6) Documentation of examination and 7) Medical decision making.

Conclusion

International multispecialty consensus was achieved on the content of a generic ultrasound rating scale. This is the first step to ensure valid assessment of clinicians in different medical specialties using ultrasound.     

Development of Self-Management Indicators for Chronic Hepatitis B Patients on Antiviral Therapy: Results of a Chinese Delphi Panel Survey

ObjectiveThis study aimed to develop a set of indicators that could be used to measure and monitor the self-management performance for chronic hepatitis B (CHB) patients on antiviral therapy in China.MethodsA two-round Delphi study via e-mail correspondence was conducted, with a group of 30 Chinese experts. The Delphi questionnaire consisted of 53 indicators identified from a literature review. Experts rated and scored the importance of indicators on a five-point Likert scale. Consensus was considered to be reached if a median score in the top tertile (4-5) and ≥80% of panel ratings in the top tertile (4-5) after Round 2. The included indicators were validated with a group of 106 CHB patients.ResultsThe response rates for the first and second rounds were 90.9% (n=30) and 86.7% (n=26), respectively. Three new indicators were suggested in the first round. 55 indicators were included in the second round after modified. 45 (81.8%) indicators achieved on the level of consensus, all of which had an inter-quartile range of 1 or below. The final set included 4 domains and 45 indicators which were well accepted and understandable by CHB patients.ConclusionThis Delphi study produced a set of 45 self-management indicators for CHB patients on antiviral therapy in China. These indicators could be used to measure and monitor the patients’ self-management performance, with the goal of improving the quality of life in this population.     

Measuring endemicity and burden of leprosy across countries and regions: A systematic review and Delphi survey

BackgroundLeprosy is a chronic infectious disease caused by Mycobacterium leprae, the annual new case detection in 2019 was 202,189 globally. Measuring endemicity levels and burden in leprosy lacks a uniform approach. As a result, the assessment of leprosy endemicity or burden are not comparable over time and across countries and regions. This can make program planning and evaluation difficult. This study aims to identify relevant metrics and methods for measuring and classifying leprosy endemicity and burden at (sub)national level.MethodsWe used a mixed-method approach combining findings from a systematic literature review and a Delphi survey. The literature search was conducted in seven databases, searching for endemicity, burden and leprosy. We reviewed the available evidence on the usage of indicators, classification levels, and scoring methods to measure and classify endemicity and burden. A two round Delphi survey was conducted to ask experts to rank and weigh indicators, classification levels, and scoring methods.ResultsThe literature review showed variation of indicators, levels, and cut-off values to measure leprosy endemicity and/or burden. The most used indicators for endemicity include new case detection rate (NCDR), new cases among children and new cases with grade 2 disability. For burden these include NCDR, MB cases, and prevalence. The classification levels ‘high’ and ‘low’ were most important. It was considered most relevant to use separate scoring methods for endemicity and burden. The scores would be derived by use of multiple indicators.ConclusionThere is great variation in the existing method for measuring endemicity and burden across countries and regions. Our findings contribute to establishing a standardized uniform approach to measure and classify leprosy endemicity and burden at (sub)national level, which would allow effective communication and planning of intervention strategies.     

A Web-Based Delphi Study for Eliciting Helpful Criteria in the Positive Diagnosis of Hemophagocytic Syndrome in Adult Patients

The diagnosis of the reactive form of hemophagocytic syndrome in adults remains particularly difficult since none of the clinical or laboratory manifestations are specific. We undertook a study in order to elicit which features constitute helpful criteria for a positive diagnosis. In this Delphi study, the features investigated in the questionnaire and the experts invited to participate in the survey were issued from a bibliographic search. The questionnaire was iteratively proposed to experts via a web-based application with a feedback of the results observed at the preceding Delphi round. Experts were asked to label each investigated criterion in one of the following categories: absolutely required, important, of minor interest, or not assessable in the routine practice environment. A positive consensus was a priori defined as at least 75% answers observed in the categories absolutely required and important. The questionnaire investigated 26 criteria and 24 experts originating from 13 countries participated in the second and final Delphi round. A positive consensus was reached for the nine following criteria: unilineage cytopenia, bicytopenia, pancytopenia, presence of hemophagocytosis pictures on a bone marrow aspirate or on a tissue biopsy, high ferritin level, fever, organomegaly, presence of a predisposing underlying disease, and high level of lactate dehydrogenase. A negative consensus was reached for 13 criteria, and an absence of consensus was observed for 4 criteria. The study constitutes the first initiative to date for defining international guidelines devoted to the positive diagnosis of the reactive form of hemophagocytic syndrome.     

Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review

OBJECTIVE: Delphi technique is a structured process commonly used to developed healthcare quality indicators, but there is a little recommendation for researchers who wish to use it. This study aimed 1) to describe reporting of the Delphi method to develop quality indicators, 2) to discuss specific methodological skills for quality indicators selection 3) to give guidance about this practice. METHODOLOGY AND MAIN FINDING: Three electronic data bases were searched over a 30 years period (1978-2009). All articles that used the Delphi method to select quality indicators were identified. A standardized data extraction form was developed. Four domains (questionnaire preparation, expert panel, progress of the survey and Delphi results) were assessed. Of 80 included studies, quality of reporting varied significantly between items (9% for year's number of experience of the experts to 98% for the type of Delphi used). Reporting of methodological aspects needed to evaluate the reliability of the survey was insufficient: only 39% (31/80) of studies reported response rates for all rounds, 60% (48/80) that feedback was given between rounds, 77% (62/80) the method used to achieve consensus and 57% (48/80) listed quality indicators selected at the end of the survey. A modified Delphi procedure was used in 49/78 (63%) with a physical meeting of the panel members, usually between Delphi rounds. Median number of panel members was 17(Q1:11; Q3:31). In 40/70 (57%) studies, the panel included multiple stakeholders, who were healthcare professionals in 95% (38/40) of cases. Among 75 studies describing criteria to select quality indicators, 28 (37%) used validity and 17(23%) feasibility. CONCLUSION: The use and reporting of the Delphi method for quality indicators selection need to be improved. We provide some guidance to the investigators to improve the using and reporting of the method in future surveys.     

A Delphi Survey and Analysis of Expert Perspectives on One Health in Australia

One Health (OH) is an interdisciplinary approach aiming to achieve optimal health for humans, animals and their environments. Case reports and systematic reviews of success are emerging; however, discussion of barriers and enablers of cross-sectoral collaboration are rare. A four-phase mixed-method Delphi survey of Australian human and animal health practitioners and policymakers (n = 52) explored areas of consensus and disagreement over: (1) the operational definition of OH; (2) potential for cross-sectoral collaboration; and (3) key priorities for shaping the development of an OH response to significantly elevated zoonotic disease risk. Participants agreed OH is essential for effective infectious disease prevention and control, and on key priorities for outbreak responses, but disagreed over definitions and the relative priority of animal health and welfare and economic considerations. Strong support emerged among Australian experts for an OH approach. There was also recognition of the need to ensure cross-sectoral differences are addressed.     

Thermographic imaging in sports and exercise medicine: A Delphi study and consensus statement on the measurement of human skin temperature

The importance of using infrared thermography (IRT) to assess skin temperature (t_sk) is increasing in clinical settings. Recently, its use has been increasing in sports and exercise medicine; however, no consensus guideline exists to address the methods for collecting data in such situations. The aim of this study was to develop a checklist for the collection of t_sk using IRT in sports and exercise medicine. We carried out a Delphi study to set a checklist based on consensus agreement from leading experts in the field. Panelists (n  =  24) representing the areas of sport science (n = 8; 33%), physiology (n = 7; 29%), physiotherapy (n = 3; 13%) and medicine (n = 6; 25%), from 13 different countries completed the Delphi process. An initial list of 16 points was proposed which was rated and commented on by panelists in three rounds of anonymous surveys following a standard Delphi procedure. The panel reached consensus on 15 items which encompassed the participants’ demographic information, camera/room or environment setup and recording/analysis of t_sk using IRT. The results of the Delphi produced the checklist entitled “Thermographic Imaging in Sports and Exercise Medicine (TISEM)” which is a proposal to standardize the collection and analysis of t_sk data using IRT. It is intended that the TISEM can also be applied to evaluate bias in thermographic studies and to guide practitioners in the use of this technique.     

Multidisciplinary Views on Applying Explicit and Implicit Motor Learning in Practice: An International Survey

Background

A variety of options and techniques for causing implicit and explicit motor learning have been described in the literature. The aim of the current paper was to provide clearer guidance for practitioners on how to apply motor learning in practice by exploring experts’ opinions and experiences, using the distinction between implicit and explicit motor learning as a conceptual departure point.

Methods

A survey was designed to collect and aggregate informed opinions and experiences from 40 international respondents who had demonstrable expertise related to motor learning in practice and/or research. The survey was administered through an online survey tool and addressed potential options and learning strategies for applying implicit and explicit motor learning. Responses were analysed in terms of consensus (≥ 70%) and trends (≥ 50%). A summary figure was developed to illustrate a taxonomy of the different learning strategies and options indicated by the experts in the survey.

Results

Answers of experts were widely distributed. No consensus was found regarding the application of implicit and explicit motor learning. Some trends were identified: Explicit motor learning can be promoted by using instructions and various types of feedback, but when promoting implicit motor learning, instructions and feedback should be restricted. Further, for implicit motor learning, an external focus of attention should be considered, as well as practicing the entire skill. Experts agreed on three factors that influence motor learning choices: the learner’s abilities, the type of task, and the stage of motor learning (94.5%; n = 34/36). Most experts agreed with the summary figure (64.7%; n = 22/34).

Conclusion

The results provide an overview of possible ways to cause implicit or explicit motor learning, signposting examples from practice and factors that influence day-to-day motor learning decisions.     

Reliability and Validity of the Chinese Version Appropriateness Evaluation Protocol

Objective

To adapt the Appropriateness Evaluation Protocol (AEP) to the specific settings of health care in China and to validate the Chinese version AEP (C-AEP).

Methods

Forward and backward translations were carried out to the original criteria. Twenty experts participated in the consultancy to form a preliminary version of the C-AEP. To ensure applicability, tests of reliability and validity were performed on 350 admissions and 3,226 hospital days of acute myocardial infraction patients and total hip replacement patients in two tertiary hospitals by two C-AEP reviewers and two physician reviewers. Overall agreement, specific agreement, and Cohen’s Kappa were calculated to compare the concordance of decisions between pairs of reviewers to test inter-rater reliability and convergent validity. The use of “overrides” and opinions of experts were recorded as measurements of content validity. Face validity was tested through collecting perspectives of nonprofessionals. Sensitivity, specificity, and predictive values were also reported.

Results

There are 14 admission and 24 days of care criteria in the initial version of C-AEP. Kappa coefficients indicate substantial agreement between reviewers: with regard to inter-rater reliability, Kappa (κ) coefficients are 0.746 (95% confidence interval [CI] 0.644–0.834) and 0.743 (95% CI 0.698–0.767) of admission and hospital days, respectively; for convergent validity, the κ statistics are 0.678 (95% CI 0.567–0.778) and 0.691 (95% CI 0.644–0.717), respectively. Overrides account for less than 2% of all judgments. Content validity and face validity were confirmed by experts and nonprofessionals, respectively. According to the C-AEP reviewers, 18.3% of admissions and 28.5% of inpatient days were deemed inappropriate.

Conclusions

The C-AEP is a reliable and valid screening tool in China’s tertiary hospitals. The prevalence of inappropriateness is substantial in our research. To reduce inappropriate utilization, further investigation is needed to elucidate the reasons and risk factors for this inappropriateness.     

What Are Priorities for Deprescribing for Elderly Patients? Capturing the Voice of Practitioners: A Modified Delphi Process

Polypharmacy and inappropriate medication use among older adults contribute to adverse drug reactions, falls, cognitive impairment, noncompliance, hospitalization and mortality. While deprescribing - tapering, reducing or stopping a medication - is feasible and relatively safe, clinicians find it difficult to carry out. Deprescribing guidelines would facilitate this process. The aim of this paper is to identify and prioritize medication classes where evidence-based deprescribing guidelines would be of benefit to clinicians. A modified Delphi approach included a literature review to identify potentially inappropriate medications for the elderly, an expert panel to develop survey content and three survey rounds to seek consensus on priorities. Panel participants included three pharmacists, two family physicians and one social scientist. Sixty-five Canadian geriatrics experts (36 pharmacists, 19 physicians and 10 nurse practitioners) participated in the survey. Twenty-nine drugs/drug classes were included in the first survey with 14 reaching the required (≥ 70%) level of consensus, and 2 new drug classes added from qualitative comments. Fifty-three participants completed round two, and 47 participants completed round three. The final five priorities were benzodiazepines, atypical antipsychotics, statins, tricyclic antidepressants, and proton pump inhibitors; nine other drug classes were also identified as being in need of evidence-based deprescribing guidelines. The Delphi consensus process identified five priority drug classes for which expert clinicians felt guidance is needed for deprescribing. The classes of drugs that emerged strongly from the rankings dealt with mental health, cardiovascular, gastroenterological, and neurological conditions. The results suggest that deprescribing and overtreatment occurs through the full spectrum of primary care, and that evidence-based deprescribing guidelines are a priority in the care of the elderly.     

Development and Preliminary Face and Content Validation of the “Which Health Approaches and Treatments Are You Using?” (WHAT) Questionnaires Assessing Complementary and Alternative Medicine Use in Pediatric Rheumatology

Objective

Complementary and alternative medicine (CAM) is commonly used by children with juvenile idiopathic arthritis (JIA), yet no validated questionnaires assess that use. The objective of this study was to develop child self- and parent proxy-report questionnaires assessing CAM use and to determine the face and content validity of the “Which Health Approaches and Treatments are you using?” (WHAT) questionnaires in pediatric rheumatology.

Methods

A sequential phased mixed methods approach was used to develop the questionnaires. A Delphi Survey of 126 experts followed by an interdisciplinary consensus conference of 14 stakeholders in CAM, general pediatrics and pediatric rheumatology was held to develop consensus on the content of the questionnaires using a nominal group technique. To determine face and content validity of the questionnaires, two groups, including (a) a purposive sample of 22 children with JIA 8 to 18 years and their parents from the Children’s Hospital of Eastern Ontario and the Hospital for Sick Children, and (b) 21 Canadian pediatric rheumatology experts, participated in interviews. Participants were independently asked about the goal, understandability and comprehensiveness of the WHAT questionnaires, as well as the relevance of items.

Results

Consensus was reached on 17 items of the WHAT questionnaires. The domains found to be relevant were child’s CAM use, factors associated with CAM use, perceived impact of CAM use, and communication about CAM. A total of 15 items in the parent proxy-report questionnaire and 13 items in the child report questionnaire showed adequate content validity.

Conclusions

Consensus was reached by experts on the content of a pediatric CAM questionnaire. Face and content validity testing and modifications made to the WHAT questionnaires have helped ensure adequate preliminary validity for use in pediatric rheumatology. This constitutes the basis for further testing of these questionnaires in pediatric rheumatology and for adaptation to other chronic diseases.     

The Rapid Growth of Fibroids during Early Pregnancy

Several studies aimed to disentangle whether pregnancy influences the growth of uterine fibroids but results were inconsistent. In this study, we speculated that fibroid enlargement during pregnancy may not be linear and we hypothesized that this phenomenon may mainly occur during initial pregnancy. To test this hypothesis, we set up a prospective cohort study of women with fibroids undergoing IVF. Cases were women achieving a viable pregnancy. Controls were the subsequent women with fibroids but failing to become pregnant. Twenty-five cases and 25 controls were recruited. The total number of fibroids in the two groups was 46 and 41, respectively. The mean ± SD diameter of the fibroids was 17±10 and 20±11 mm, respectively (p = 0.18). A statistically significant enlargement emerged exclusively in pregnant women. The median (Interquartile Range) modification of the diameter of the lesions in cases and controls was +34% (+6%/+65%) and +2% (−6%/+12%), respectively (p<0.001). The median (Interquartile Range) modification of the volume of the lesions was +140% (+23%/+357%) and 0% (−18%/+37%), respectively (p<0.001). In pregnant women, we failed to document any significant correlation between the magnitude of the growth and ovarian responsiveness to hyper-stimulation, suggesting that steroids hormones are not the unique factors involved. In conclusion, fibroids undergo a rapid and remarkable growth during initial pregnancy. Reasons behind this phenomenon remain to be clarified. The early rise in steroids hormones during early pregnancy may not be sufficient to explain the process. Other pregnancy-related hormones and proteins may play also key roles.     

Lymphatic filariasis in 2016 in American Samoa: Identifying clustering and hotspots using non-spatial and three spatial analytical methods

BackgroundAmerican Samoa completed seven rounds of mass drug administration from 2000–2006 as part of the Global Programme to Eliminate Lymphatic Filariasis (LF). However, resurgence was confirmed in 2016 through WHO-recommended school-based transmission assessment survey and a community-based survey. This paper uses data from the 2016 community survey to compare different spatial and non-spatial methods to characterise clustering and hotspots of LF.MethodNon-spatial clustering of infection markers (antigen [Ag], microfilaraemia [Mf], and antibodies (Ab [Wb123, Bm14, Bm33]) was assessed using intra-cluster correlation coefficients (ICC) at household and village levels. Spatial dependence, clustering and hotspots were examined using semivariograms, Kulldorf’s scan statistic and Getis-Ord Gi* statistics based on locations of surveyed households.ResultsThe survey included 2671 persons (750 households, 730 unique locations in 30 villages). ICCs were higher at household (0.20–0.69) than village levels (0.10–0.30) for all infection markers. Semivariograms identified significant spatial dependency for all markers (range 207–562 metres). Using Kulldorff’s scan statistic, significant spatial clustering was observed in two previously known locations of ongoing transmission: for all markers in Fagali’i and all Abs in Vaitogi. Getis-Ord Gi* statistic identified hotspots of all markers in Fagali’i, Vaitogi, and Pago Pago-Anua areas. A hotspot of Ag and Wb123 Ab was identified around the villages of Nua-Seetaga-Asili. Bm14 and Bm33 Ab hotspots were seen in Maleimi and Vaitogi-Ili’ili-Tafuna.ConclusionOur study demonstrated the utility of different non-spatial and spatial methods for investigating clustering and hotspots, the benefits of using multiple infection markers, and the value of triangulating results between methods.     

Likely health outcomes for untreated acute febrile illness in the tropics in decision and economic models; a Delphi survey

Background

Modelling is widely used to inform decisions about management of malaria and acute febrile illnesses. Most models depend on estimates of the probability that untreated patients with malaria or bacterial illnesses will progress to severe disease or death. However, data on these key parameters are lacking and assumptions are frequently made based on expert opinion. Widely diverse opinions can lead to conflicting outcomes in models they inform.

Methods and Findings

A Delphi survey was conducted with malaria experts aiming to reach consensus on key parameters for public health and economic models, relating to the outcome of untreated febrile illnesses. Survey questions were stratified by malaria transmission intensity, patient age, and HIV prevalence. The impact of the variability in opinion on decision models is illustrated with a model previously used to assess the cost-effectiveness of malaria rapid diagnostic tests. Some consensus was reached around the probability that patients from higher transmission settings with untreated malaria would progress to severe disease (median 3%, inter-quartile range (IQR) 1–5%), and the probability that a non-malaria illness required antibiotics in areas of low HIV prevalence (median 20%). Children living in low transmission areas were considered to be at higher risk of progressing to severe malaria (median 30%, IQR 10–58%) than those from higher transmission areas (median 13%, IQR 7–30%). Estimates of the probability of dying from severe malaria were high in all settings (medians 60–73%). However, opinions varied widely for most parameters, and did not converge on resurveying.

Conclusions

This study highlights the uncertainty around potential consequences of untreated malaria and bacterial illnesses. The lack of consensus on most parameters, the wide range of estimates, and the impact of variability in estimates on model outputs, demonstrate the importance of sensitivity analysis for decision models employing expert opinion. Results of such models should be interpreted cautiously. The diversity of expert opinion should be recognised when policy options are debated.     

Stakeholder involvement in establishing a milk quality sub-index in dairy cow breeding goals: a Delphi approach

The relative weighting on traits within breeding goals are generally determined by bio-economic models or profit functions. While such methods have generally delivered profitability gains to producers, and are being expanded to consider non-market values, current approaches generally do not consider the numerous and diverse stakeholders that affect, or are affected, by such tools. Based on principles of respondent anonymity, iteration, controlled feedback and statistical aggregation of feedback, a Delphi study was undertaken to gauge stakeholder opinion of the importance of detailed milk quality traits within an overall dairy breeding goal for profit, with the aim of assessing its suitability as a complementary, participatory approach to defining breeding goals. The questionnaires used over two survey rounds asked stakeholders: (a) their opinion on incorporating an explicit sub-index for milk quality into a national breeding goal; (b) the importance they would assign to a pre-determined list of milk quality traits and (c) the (relative) weighting they would give such a milk quality sub-index. Results from the survey highlighted a good degree of consensus among stakeholders on the issues raised. Similarly, revelation of the underlying assumptions and knowledge used by stakeholders to make their judgements illustrated their ability to consider a range of perspectives when evaluating traits, and to reconsider their answers based on the responses and rationales given by others, which demonstrated social learning. Finally, while the relative importance assigned by stakeholders in the Delphi survey (4% to 10%) and the results of calculations based on selection index theory of the relative emphasis that should be placed on milk quality to halt any deterioration (16%) are broadly in line, the difference indicates the benefit of considering more than one approach to determining breeding goals. This study thus illustrates the role of the Delphi technique, as a complementary approach to traditional approaches, to defining breeding goals. This has implications for how breeding goals will be defined and in determining who should be involved in the decision-making process.     

Experts and Novices Use the Same Factors–But Differently–To Evaluate Pearl Quality

Well-trained experts in pearl grading have been thought to evaluate pearls according to their glossiness, interference color, and shape. However, the characteristics of their evaluations are not fully understood. Using pearl grading experiments, we investigate the consistency of novice (i.e., without knowledge of pearl grading) and expert participants’ pearl grading skill and then compare the novices’ grading with that of experts; furthermore, we discuss the relationship between grading, interference color, and glossiness. We found that novices’ grading was significantly less concordant with experts average grading than was experts’ grading; more than half of novices graded pearls the opposite of how experts graded those same pearls. However, while experts graded pearls more consistently than novices did, novices’ consistency was relatively high. We also found differences between the groups in regression analyses that used interference color and glossiness as explanatory variables and were conducted for each trial. Although the regression coefficient was significant in 60% of novices’ trials, there were fewer significant trials for the experts (20%). This indicates that novices can also make use of these two factors, but that their usage is simpler than that of the experts. These results suggest that experts and novices share some values about pearls but that the evaluation method is elaborated for experts.