Use of Expert Panels to Define the Reference Standard in Diagnostic Research: A Systematic Review of Published Methods and Reporting

Background

In diagnostic studies, a single and error-free test that can be used as the reference (gold) standard often does not exist. One solution is the use of panel diagnosis, i.e., a group of experts who assess the results from multiple tests to reach a final diagnosis in each patient. Although panel diagnosis, also known as consensus or expert diagnosis, is frequently used as the reference standard, guidance on preferred methodology is lacking. The aim of this study is to provide an overview of methods used in panel diagnoses and to provide initial guidance on the use and reporting of panel diagnosis as reference standard.

Methods and Findings

PubMed was systematically searched for diagnostic studies applying a panel diagnosis as reference standard published up to May 31, 2012. We included diagnostic studies in which the final diagnosis was made by two or more persons based on results from multiple tests. General study characteristics and details of panel methodology were extracted. Eighty-one studies were included, of which most reported on psychiatry (37%) and cardiovascular (21%) diseases. Data extraction was hampered by incomplete reporting; one or more pieces of critical information about panel reference standard methodology was missing in 83% of studies. In most studies (75%), the panel consisted of three or fewer members. Panel members were blinded to the results of the index test results in 31% of studies. Reproducibility of the decision process was assessed in 17 (21%) studies. Reported details on panel constitution, information for diagnosis and methods of decision making varied considerably between studies.

Conclusions

Methods of panel diagnosis varied substantially across studies and many aspects of the procedure were either unclear or not reported. On the basis of our review, we identified areas for improvement and developed a checklist and flow chart for initial guidance for researchers conducting and reporting of studies involving panel diagnosis. Please see later in the article for the Editors'' Summary     

Primary cecal lymphoma. Report of a case with preoperative diagnosis by fine needle aspiration and immunocytochemistry.

Fine needle aspiration of a cecal mass was performed on a patient with a cecal tumor and iron-deficiency anemia. Cytologic studies of the air-dried smears showed large cell lymphoma. The diagnosis of large cell lymphoma of the B-cell type was affirmed by immunocytochemical studies and at laparotomy and resection of the tumor. Even under unusual circumstances, the diagnosis of such a rare abdominal lesion as primary cecal lymphoma can be made with certainty by cytologic and immunologic studies of fine needle aspirates.     

Ancillary alpha-methylacyl-CoA racemase immunocytochemistry in the diagnosis of adenocarcinoma of the prostate in urinary cytology: a case report

BACKGROUND: Alpha-methylacyl-coA racemase (AMACR) was recently shown to be a sensitive immunohistochemical marker for substantiating a diagnosis of adenocarcinoma of the prostate. Its applicability to exfoliative urinary cytology has not been investigated before. CASE: The patient was a 77-year-old male with a history of persistent intermittent hematuria who was evaluated with urine cytology. Cytopathologic studies were interpreted to demonstrate "atypical urothelial cells." The patient was evaluated with computed tomography, which demonstrated new left hydroureter and hydronephrosis. He underwent cystoscopy, which showed an abnormal left hemitrigone with the left ureteral orifice obscured by an infiltrative mass. This area was biopsied, and histologic studies of the tissue chips demonstrated underlying prostatic adenocarcinoma directly invading the urothelium. We used AMACR immunoreactivity on a retrospectively studied, catheterized urine slide to confirm the diagnosis. CONCLUSION: This case suggests that combined cytomorphology and immunocytochemisty for AMACR may allow an accurate identification of cells of prostatic adenocarcinoma when cytomorphologic studies or the clinical history raises the differential diagnosis of prostate cancer presenting with exfoliation of malignant cells into the urine.     

Performance of Polymerase Chain Reaction Analysis of the Amniotic Fluid of Pregnant Women for Diagnosis of Congenital Toxoplasmosis: A Systematic Review and Meta-Analysis

IntroductionCongenital infection caused by Toxoplasma gondii can cause serious damage that can be diagnosed in utero or at birth, although most infants are asymptomatic at birth. Prenatal diagnosis of congenital toxoplasmosis considerably improves the prognosis and outcome for infected infants. For this reason, an assay for the quick, sensitive, and safe diagnosis of fetal toxoplasmosis is desirable.GoalTo systematically review the performance of polymerase chain reaction (PCR) analysis of the amniotic fluid of pregnant women with recent serological toxoplasmosis diagnoses for the diagnosis of fetal toxoplasmosis.MethodA systematic literature review was conducted via a search of electronic databases; the literature included primary studies of the diagnostic accuracy of PCR analysis of amniotic fluid from pregnant women who seroconverted during pregnancy. The PCR test was compared to a gold standard for diagnosis.ResultsA total of 1.269 summaries were obtained from the electronic database and reviewed, and 20 studies, comprising 4.171 samples, met the established inclusion criteria and were included in the review. The following results were obtained: studies about PCR assays for fetal toxoplasmosis are generally susceptible to bias; reports of the tests’ use lack critical information; the protocols varied among studies; the heterogeneity among studies was concentrated in the tests’ sensitivity; there was evidence that the sensitivity of the tests increases with time, as represented by the trimester; and there was more heterogeneity among studies in which there was more time between maternal diagnosis and fetal testing. The sensitivity of the method, if performed up to five weeks after maternal diagnosis, was 87% and specificity was 99%.ConclusionThe global sensitivity heterogeneity of the PCR test in this review was 66.5% (I²). The tests show low evidence of heterogeneity with a sensitivity of 87% and specificity of 99% when performed up to five weeks after maternal diagnosis. The test has a known performance and could be recommended for use up to five weeks after maternal diagnosis, when there is suspicion of fetal toxoplasmosis.     

Prenatal diagnosis in a familial form of male pseudohermaphroditism due to 17-keto reductase deficiency

In an infant considered at birth as a female but with easily palpable gonads in the labia major, the XY karyotype and the endocrine studies (determination of plasma levels of steroid hormones under basal conditions and during hCG stimulation) were consistent with the diagnosis of male pseudohermaphroditism due to 17-keto reductase deficiency. During the second pregnancy an amniocentesis revealed a 46 XY karyotype. Endocrine studies performed on the amniotic fluid at midgestation suggested that the fetus was affected by the same enzyme defect. After birth, the diagnosis was demonstrated with anatomical an endocrine studies.     

Immunocytochemical and ultrastructural findings in a case of rare carcinoma of the pancreas with predominance of malignant squamous cells in an intraoperative needle aspirate

The cytologic diagnosis by intraoperative needle aspiration of a rare carcinoma of the pancreas composed predominantly of malignant squamous cells is reported. Both poorly differentiated and keratinized well-differentiated squamous cells were seen in the aspirate. Tissue for histologic confirmation was not obtained because of the position of the mass. Confirmatory ultrastructural and immunoperoxidase studies were thus done on the aspirated material to confirm the diagnosis.     

Comparative ultrasound and magnetic resonance diagnosis of fetal CNS malformations

The study was undertaken to optimize the diagnosis of fetal CNS and facial malformations, by using a complex of ultrasound (US) and magnetic resonance imaging (MRI) studies. A hundred and forty-four fetuses with suspected CNS and facial malformations were examined. The US study conducted by a specially developed protocol was supplemented by MRI (48 fetuses) also made by a specially developed protocol. Various fetal CNS malformations, such as neural tube defects, congenital endbrain malformations, cystic lesions, tumors, ventricular complex anomalies, defects of the face and eyes, multiple defects, including CNS and facial anomalies, were detected. With MRI, the diagnosis was changed in 33% of cases. The application of a complex of US and MRI studies enhances the efficiency of diagnosis of congenital CNS and facial malformations in the fetus. MRI in the diagnosis of fetal CNS and facial malformations has a number of advantages and should be used if there is some difficulty in establishing a diagnosis when an US study is performed.     

Reliability of telepathology for frozen section service.

One of the most promising applications of telepathology (pathology at a distance by electronic transmission of images in pathology) is frozen section diagnosis, especially because by means of this tool operations requiring an intraoperative histopathological diagnosis are feasible at hospitals without a pathologist on-site. For the introduction of this diagnostic tool into pathologist's daily practice the evidence of its diagnostic accuracy comparable to that of the conventional frozen section diagnosis is crucial. For this purpose the literature on the diagnostic accuracy of telepathological frozen section diagnosis was reviewed. In a metaanalysis these studies and reports, in which a total of more than 1290 cases had been examined, showed a slightly lower overall diagnostic accuracy (of the telepathological frozen section diagnosis) of about 0.91 than the conventional frozen section diagnosis with an average accuracy of about 0.98 found in an analysis of several studies (on frozen section diagnosis of different organs). This difference is at least predominantly caused by a higher rate of deferred and false negative frozen section diagnoses in the telepathological method, while the specificity of both methods, each more than 0.99 was not significantly different. In conclusion, the introduction of a telepathological frozen section diagnosis for hospitals without an acceptable access to a pathologist is justifiable already at the current state of the technological development especially when considering the advantages (time saving, reduction in costs) compared to the alternative of surgical interventions without access to an intraoperative diagnosis.     

Carcinoid tumour of male breast diagnosed by fine needle aspiration

A primary carcinoid tumour of the breast in a 66-year-old man was diagnosed by fine-needle aspiration cytology. The nature of the lesion was proved by histochemical and immunocytochemical studies. The importance of a conclusive diagnosis is discussed and the value of immunocytochemical analysis as an aid to cytomorphologic diagnosis is demonstrated.     

DNA-cytometric diagnosis of prospective malignancy in borderline lesions of the uterine cervix

Interactive DNA cytometry was used for the diagnosis of prospective malignancy in 48 smears with borderline lesions (mild and moderate dysplasias) of the uterine cervix. In addition, 183 smears with benign squamous epithelia, 38 with carcinoma in situ and 7 with invasive squamous carcinoma were also measured. Nuclear Feulgen-DNA measurements were performed using various methods, and the resulting data were analyzed by an algorithm for a DNA-cytophotometric diagnosis of malignancy. The results were compared with the data on follow-up and subsequent histologic studies in these cases. There was no false-positive diagnosis in the 183 benign smears and only 1 false-negative diagnosis in the 76 histologically proven squamous-cell carcinomas, which yields a specificity of 100% and a sensitivity of 98.6%. The sensitivity for the detection of subsequent histologically proven malignancy in cases with cytologically mild or moderate dysplasia amounted to 97%. In 13 borderline cases, there was a mean interval of 21 months between the taking of the cytologic smear on which the DNA diagnosis of malignancy was made and the date on which the histologic confirmation of malignancy was made. In 17% of the cytologically dysplastic cases, the DNA diagnosis of malignancy was not verified by subsequent histologic investigation. These results indicate that interactive DNA cytometry is able to detect prospective malignancy in smears from borderline lesions of the uterine cervix with a high sensitivity.     

Extraskeletal Ewing's sarcoma. Diagnosis of a case by fine needle aspiration cytology

A retroperitoneal mass was subjected to fine needle aspiration biopsy for cytologic evaluation. The aspirate consisted of a monotonous population of undifferentiated tumor cells whose cytologic appearance was consistent with a small-cell (Ewing's) sarcoma. The differential diagnosis of extraskeletal Ewing's sarcoma from other small-cell neoplasms, such as lymphoma, neuroblastoma and other soft-tissue sarcoma, cannot be made solely on the basis of morphologic studies. As in this case, histochemical studies and consideration of the clinical features are needed to make the final diagnosis.     

交感皮肤反应在糖尿病周围神经病变诊断中的应用

糖尿病周围神经病变（Diabetic Peripheral Neuropathy,DPN）是糖尿病最常见的并发症之一。由于目前DPN发病机制不清楚、治疗效果不理想,故早诊断、早治疗显得至关重要。有研究指出交感神经皮肤反应（Sympathetic Skin Response,SSR）可作为评价2型糖尿病患者早期周围植物神经功能状态的指标。本文对SSR应用于DPN临床诊断中的检测技术、观测参数进行综述,发现多数研究中指出,在临床应用中SSR波形、波幅以及潜伏期指标的异常率常受到多种因素的影响、会发生很大波动,而电位曲线下面积减少值相对稳定。据此笔者建议在DPN早期临床诊断中以SSR电位曲线下面积减少作为关键参数,辅助参考SSR波形、波幅以及潜伏期指标进行诊断。     

Fine needle aspiration cytology of metastatic lymphoepithelioma. A case report

A case is presented of lymphoepithelioma (undifferentiated nasopharyngeal carcinoma) metastatic to the cervical lymph nodes in a 12-year-old boy for whom material was obtained by fine needle aspiration (FNA) for the primary diagnosis as well as for ancillary studies. Papanicolaou-stained smears demonstrated the characteristic cytopathologic features of Regaud-type lymphoepithelioma; the diagnosis was substantiated by immunocytochemical and electron microscopic studies. This report discusses the reliability and rapidity of FNA in definitively diagnosing undifferentiated metastatic malignancies as well as providing superior material for ancillary studies demanded by lesions with complicated and difficult differential diagnoses.     

Spiral computed tomography in the diagnosis of the abdomen gunshot wounds and their complications

The results of studies by spiral computed tomography (SCT) in 62 patients with abdominal wounds were analyzed at the Main Military Clinical Hospital of Internal Forces, Ministry of Internal Affairs of Russia. The main contingent of the wounded included active servicemen who had been wounded in the areas of local conflicts, military men. The studies were conducted on days 2 to 15 of their receiving a wound. The potentialities of the technique in evaluating damages to abdominal organs, retroabdominal space, bony structures, in assessing the topography of foreign bodies are described. SCT was compared with classical X-ray and ultrasound studies, laparocentesis which are traditionally used in the diagnosis of gunshot injury. In the authors' opinion, SCT has become the method of choice in the diagnosis of gunshot injury to the abdomen and its complications today.     

Fine needle aspiration cytology of pulmonary infarct

The cytologic features of a pulmonary infarct diagnosed by fine needle aspiration (FNA) cytology are reported in a 54-year-old white man with a prior history of renal transplant surgery. Although the diagnosis of pulmonary embolus was unsuspected, FNA cytology suggested the correct diagnosis, which was confirmed by subsequent radiologic studies. This is believed to be the first reported case of pulmonary infarct diagnosed by FNA cytology. Cytologic features of the entity are discussed along with potential sources for a false-positive diagnosis of malignancy.     

To differential diagnosis of esophageal cancer and other diseases with the similar clinical symptoms, by using traditional X-ray study

The principal purpose of the paper is to assess the role of the X-ray section of current radiation diagnosis in obtaining objective information and interpreting it in esophageal cancer. The study was methodically based on its serial studies, by comparing the data of clinical, radiation, and endoscopic diagnosis with those of morphological studies of the biopsy specimens taken at resection for cancer of the esophagus and the upper stomach, and histologically examined them in diseases running with the similar clinical symptoms. The complex of radiation techniques included traditional X-ray study (compact filling, double contrasting). A specially developed procedure that allowed physicians the opportunity to examine the cardioesophageal area in the fragmentary fashion was used during morphological studies. In cancer of the upper stomach, the lower esophageal segments were also involved in the process in 79.9%. It should be also noted that this number of observations includes some cases that clinical symptomatology and endoscopy strongly suggest cancer of the lower third of the esophagus. The authors have arrived at the conclusion that the present-day X-ray semiotics of esophageal diseases running with the similar clinical symptomatology permits their differential diagnosis just before the use of endoscopy and histology of biopsy specimens and that the unique advantage of radiation diagnosis over endoscopy is to specify the initial site of a tumor, namely, to detect primary gastric damage in cardioesophageal carcinoma.     

Role of digital X-ray study in the current diagnosis of gastric cancer

The authors attempted to assess and provide evidence for the expedience of using a digital radiography CR system in clinical gastroenterology. The prerequisite for this was the results of large-scale diagnostic studies of different organ and systemic diseases. The authors underline the specific features of application of this digital system: the latter allows several analogue X-ray apparatuses to be transformed to digital ones, provides economic efficiency as compared with apparatuses with the direct digitization of an image, shows telemedical prospects, and has low radiation loads as evidenced by research and experimental studies even in comparison with green systems, which makes the use of these digital systems in X-ray gastroenterology highly tempting. Based on a great body of data from 126 studies, the authors could show the effectiveness of the digital radiography CR system in the diagnosis of gastric cancer, demonstrate a modernized approach to formatting an obtained digital image, the possibility of postprocessor treatment that can enhance the validity of existing X-ray symptoms. This all permitted the authors to recommend using the CR systems in the diagnosis of gastric cancer, by taking into account the current obvious predominance of its morphological diffuse and mixed forms and corresponding difficulties of their endoscopic diagnosis.     

Value of muscle studies in the early diagnosis of Schwartz-Jampel syndrome

The authors report a case of Schwartz-Jampel syndrome (osteo-chondro muscular dystrophy with myotonia). The diagnosis was made when the child was 3 1/2 year old. Then, there were no clinical symptoms; however, the electromyographic and histologic patterns of the disease were found. Two years later, the clinical status provided confirmation of the diagnosis. The discussion focuses on the difficulty of the diagnosis and the relevance of electrophysiological studies and muscular biopsy in order to distinguish this disease from others with similar clinical pattern (as Freeman-Sheldon, or Marden Walker syndromes).     

Laparoscopic studies of ovulation, pregnancy diagnosis, and follicle aspiration in sheep

A technique for rapid and repeated laparoscopic ovarian examination was developed for use in the sheep. A midventral approach was used on ewes under general anesthesia on a 45-degree sloped table. Examination could be done as often as every other day.The technique was used for detection of ovulation in studies of gonadotropin stimulation in anestrous ewes. Ovulation was induced with 5 to 10 mg FSH and 100 to 2000 iu HCG. The use of laparoscopy in pregnancy diagnosis was also investigated. Accurate diagnosis could be made by Day 17 using both uterine and ovarian characteristics. Visualization through the laparoscope was utilized for aspiration of follicular oocytes using a 23 ga, 11 cm needle.     

Use of Anti-Granulocyte Scintigraphy with 99mTc-Labeled Monoclonal Antibodies for the Diagnosis of Periprosthetic Infection in Patients after Total Joint Arthroplasty: A Diagnostic Meta-Analysis

The accurate diagnosis of periprosthetic joint infections (PJI) is crucial for therapy and the prevention of complications. No diagnostic test of PJI is 100% accurate. The aim of this study was to assess the use of anti-granulocyte scintigraphy using 99 mTc-labeled monoclonal antibodies to diagnose PJI after total joint arthroplasty. A systematic search of all relevant studies published through January 2013 was conducted using the MEDLINE, EMBASE, OVID, and ScienceDirect databases. Observational studies that assessed the accuracy of the anti-granulocyte scintigraphy with monoclonal antibodies or antibody fragments labeled with technetium 99 m in diagnosis for PJI and provided data on specificity and sensitivity were identified. Standard methods recommended for meta-analysis of diagnostic accuracy were used. Nineteen studies were eligible for inclusion. The results demonstrated that the area under the summary receiver operator curve was 0.88, and the diagnostic accuracy (Q*) was 0.81. Additionally, the diagnostic odds ratio (DOR) was 18.76 with a corresponding 95% confidence interval of 10.45–33.68. The pooled sensitivity and specificity of the diagnostic method for the diagnosis of PJI were 83% and 79%, respectively, while the pooled positive likelihood ratio (PLR) was 3.56, and the negative likelihood ratio (NLR) was 0.26. Anti-granulocyte scintigraphy using 99 mTc-labeled monoclonal antibodies has a reasonable role in the diagnosis of PJI after total joint arthroplasty. Due to the limitations of the present meta-analysis, additional high-quality original studies are required to confirm the predictive value.