The Development and Validation of the Osteoporosis Prevention and Awareness Tool (OPAAT) in Malaysia

ObjectivesTo develop and validate Osteoporosis Prevention and Awareness Tool (OPAAT) in Malaysia.MethodsThe OPAAT was modified from the Malaysian Osteoporosis Knowledge Tool and developed from an exploratory study on patients. Face and content validity was established by an expert panel. The OPAAT consists of 30 items, categorized into three domains. A higher score indicates higher knowledge level. English speaking non-osteoporotic postmenopausal women ≥50 years of age and pharmacists were included in the study.ResultsA total of 203 patients and 31 pharmacists were recruited. Factor analysis extracted three domains. Flesch reading ease was 59.2. The mean±SD accuracy rate was 0.60±0.22 (range: 0.26-0.94). The Cronbach’s α for each domain ranged from 0.286-0.748. All items were highly correlated (Spearman’s rho: 0.761-0.990, p<0.05), with no significant change in the overall test-retest scores, indicating that OPAAT has achieved stable reliability. Pharmacists had higher knowledge score than patients (80.9±8.7vs63.6±17.4, p<0.001), indicating that the OPAAT was able to discriminate between the knowledge levels of pharmacists and patients.ConclusionThe OPAAT was found to be a valid and reliable instrument for assessing patient’s knowledge about osteoporosis and its prevention in Malaysia. The OPAAT can be used to identify individuals in need of osteoporosis educational intervention.     

Development and Validation of the Chinese Attitudes to Starting Insulin Questionnaire (Ch-ASIQ) for Primary Care Patients with Type 2 Diabetes

Objectives

To develop and evaluate the psychometric properties of a Chinese questionnaire which assesses the barriers and enablers to commencing insulin in primary care patients with poorly controlled Type 2 diabetes.

Research Design and Method

Questionnaire items were identified using literature review. Content validation was performed and items were further refined using an expert panel. Following translation, back translation and cognitive debriefing, the translated Chinese questionnaire was piloted on target patients. Exploratory factor analysis and item-scale correlations were performed to test the construct validity of the subscales and items. Internal reliability was tested by Cronbach’s alpha.

Results

Twenty-seven identified items underwent content validation, translation and cognitive debriefing. The translated questionnaire was piloted on 303 insulin naïve (never taken insulin) Type 2 diabetes patients recruited from 10 government-funded primary care clinics across Hong Kong. Sufficient variability in the dataset for factor analysis was confirmed by Bartlett’s Test of Sphericity (P<0.001). Using exploratory factor analysis with varimax rotation, 10 factors were generated onto which 26 items loaded with loading scores > 0.4 and Eigenvalues >1. Total variance for the 10 factors was 66.22%. Kaiser-Meyer-Olkin measure was 0.725. Cronbach’s alpha coefficients for the first four factors were ≥0.6 identifying four sub-scales to which 13 items correlated. Remaining sub-scales and items with poor internal reliability were deleted. The final 13-item instrument had a four scale structure addressing: ‘Self-image and stigmatization’; ‘Factors promoting self-efficacy; ‘Fear of pain or needles’; and ‘Time and family support’.

Conclusion

The Chinese Attitudes to Starting Insulin Questionnaire (Ch-ASIQ) appears to be a reliable and valid measure for assessing barriers to starting insulin. This short instrument is easy to administer and may be used by healthcare providers and researchers as an assessment tool for Chinese diabetic primary care patients, including the elderly, who are unwilling to start insulin.     

Psychometric Properties of the Japanese Version of the STarT Back Tool in Patients with Low Back Pain

Background and Objective

The STarT Back Tool uses prognostic indicators to classify patients with low back pain into three risk groups to guide early secondary prevention in primary care. The present study aimed to evaluate the psychometric properties of the Japanese version of the tool (STarT-J).

Methods

An online survey was conducted among Japanese patients with low back pain aged 20–64 years. Reliability was assessed by examining the internal consistency of the overall and psychosocial subscales using Cronbach’s alpha coefficients. Spearman’s correlation coefficients were used to evaluate the concurrent validity between the STarT-J total score/psychosocial subscore and standard reference questionnaires. Discriminant validity was evaluated by calculating the area under the curves (AUCs) for the total and psychosocial subscale scores against standard reference cases. Known-groups validity was assessed by examining the relationship between low back pain-related disability and STarT-J scores.

Results

The analysis included data for 2000 Japanese patients with low back pain; the mean (standard deviation [SD]) age was 47.7 (9.3) years, and 54.1% were male. The mean (SD) STarT-J score was 2.2 (2.1). The Cronbach’s alpha coefficient was 0.75 for the overall scale and 0.66 for the psychosocial subscale. Spearman’s correlation coefficients ranged from 0.30 to 0.59, demonstrating moderate to strong concurrent validity. The AUCs for the total score ranged from 0.65 to 0.83, mostly demonstrating acceptable discriminative ability. For known-groups validity, participants with more somatic symptoms had higher total scores. Those in higher STarT-J risk groups had experienced more low back pain-related absences.

Conclusions

The overall STarT-J scale was internally consistent and had acceptable concurrent, discriminant, and known-groups validity. The STarT-J can be used with Japanese patients with low back pain.     

Validation of the Chinese Version of the Quality of Nursing Work Life Scale

Quality of Nursing Work Life (QNWL) serves as a predictor of a nurse’s intent to leave and hospital nurse turnover. However, QNWL measurement tools that have been validated for use in China are lacking. The present study evaluated the construct validity of the QNWL scale in China. A cross-sectional study was conducted conveniently from June 2012 to January 2013 at five hospitals in Guangzhou, which employ 1938 nurses. The participants were asked to complete the QNWL scale and the World Health Organization Quality of Life abbreviated version (WHOQOL-BREF). A total of 1922 nurses provided the final data used for analyses. Sixty-five nurses from the first investigated division were re-measured two weeks later to assess the test-retest reliability of the scale. The internal consistency reliability of the QNWL scale was assessed using Cronbach’s α. Test-retest reliability was assessed using the intra-class correlation coefficient (ICC). Criterion-relation validity was assessed using the correlation of the total scores of the QNWL and the WHOQOL-BREF. Construct validity was assessed with the following indices: χ² statistics and degrees of freedom; relative mean square error of approximation (RMSEA); the Akaike information criterion (AIC); the consistent Akaike information criterion (CAIC); the goodness-of-fit index (GFI); the adjusted goodness of fit index; and the comparative fit index (CFI). The findings demonstrated high internal consistency (Cronbach’s α = 0.912) and test-retest reliability (interclass correlation coefficient = 0.74) for the QNWL scale. The chi-square test (χ² = 13879.60, df [degree of freedom] = 813 P = 0.0001) was significant. The RMSEA value was 0.091, and AIC = 1806.00, CAIC = 7730.69, CFI = 0.93, and GFI = 0.74. The correlation coefficient between the QNWL total scores and the WHOQOL-BREF total scores was 0.605 (p<0.01). The QNWL scale was reliable and valid in Chinese-speaking nurses and could be used as a clinical and research instrument for measuring work-related factors among nurses in China.     

Development of Two Barthel Index-Based Supplementary Scales for Patients with Stroke

Background

The Barthel Index (BI) assesses actual performance of activities of daily living (ADL). However, comprehensive assessment of ADL functions should include two other constructs: self-perceived difficulty and ability.

Objective

The aims of this study were to develop two BI-based Supplementary Scales (BI-SS), namely, the Self-perceived Difficulty Scale and the Ability Scale, and to examine the construct validity of the BI-SS in patients with stroke.

Method

The BI-SS was first developed by consultation with experts and then tested on patients to confirm the clarity and feasibility of administration. A total of 306 participants participated in the construct validity study. Construct validity was investigated using Mokken scale analysis and analyzing associations between scales. The agreement between each pair of the scales’ scores was further examined.

Results

The Self-perceived Difficulty Scale consisted of 10 items, and the Ability Scale included 8 items (excluding both bladder and bowel control items). Items in each individual scale were unidimensional (H≥0.5). The scores of the Self-perceived Difficulty and Ability Scales were highly correlated with those of the BI (rho = 0.78 and 0.90, respectively). The scores of the two BI-SS scales and BI were significantly different from each other (p<.001). These results indicate that both BI-SS scales assessed unique constructs.

Conclusions

The BI-SS had overall good construct validity in patients with stroke. The BI-SS can be used as supplementary scales for the BI to comprehensively assess patients’ ADL functions in order to identify patients’ difficulties in performing ADL tasks, plan intervention strategies, and assess outcomes.     

Measuring Professionalism in Medicine and Nursing: Results of a European Survey

Background

Leveraging professionalism has been put forward as a strategy to drive improvement of patient care. We investigate professionalism as a factor influencing the uptake of quality improvement activities by physicians and nurses working in European hospitals.

Objective

To (i) investigate the reliability and validity of data yielded by using the self-developed professionalism measurement tool for physicians and nurses, (ii) describe their levels of professionalism displayed, and (iii) quantify the extent to which professional attitudes would predict professional behaviors.

Methods and Materials

We designed and deployed survey instruments amongst 5920 physicians and nurses working in European hospitals. This was conducted under the cross-sectional multilevel study “Deepening Our Understanding of Quality Improvement in Europe” (DUQuE). We used psychometric and generalized linear mixed modelling techniques to address the aforementioned objectives.

Results

In all, 2067 (response rate 69.8%) physicians and 2805 nurses (94.8%) representing 74 hospitals in 7 European countries participated. The professionalism instrument revealed five subscales of professional attitude and one scale for professional behaviour with moderate to high internal consistency and reliability. Physicians and nurses display equally high professional attitude sum scores (11.8 and 11.9 respectively out of 16) but seem to have different perceptions towards separate professionalism aspects. Lastly, professionals displaying higher levels of professional attitudes were more involved in quality improvement actions (physicians: b = 0.019, P<0.0001; nurses: b = 0.016, P<0.0001) and more inclined to report colleagues’ underperformance (physicians – odds ratio (OR) 1.12, 95% CI 1.01–1.24; nurses – OR 1.11, 95% CI 1.01–1.23) or medical errors (physicians – OR 1.14, 95% CI 1.01–1.23; nurses – OR 1.43, 95% CI 1.22–1.67). Involvement in QI actions was found to increase the odds of reporting incompetence or medical errors.

Conclusion

A tool that reliably and validly measures European physicians’ and nurses’ commitment to professionalism is now available. Collectively leveraging professionalism as a quality improvement strategy may be beneficial to patient care quality.     

Knowledge,Practices, and Perceived Barriers Regarding Cancer Pain Management Among Physicians and Nurses In Korea: A Nationwide Multicenter Survey

Purpose

Medical professionals’ practices and knowledge regarding cancer pain management have often been cited as inadequate. This study aimed to evaluate knowledge, practices and perceived barriers regarding cancer pain management among physicians and nurses in Korea.

Methods

A nationwide questionnaire survey was administered to physicians and nurses involved in the care of cancer patients. Questionnaire items covered pain assessment and documentation practices, knowledge regarding cancer pain management, the perceived barriers to cancer pain control, and processes perceived as the major causes of delay in opioid administration.

Results

A total of 333 questionnaires (149 physicians and 284 nurses) were analyzed. Nurses performed pain assessment and documentation more regularly than physicians did. Although physicians had better knowledge of pain management than did nurses, both groups lacked knowledge regarding the side effects and pharmacology of opioids. Physicians working in the palliative care ward and nurses who had received pain management education obtained higher scores on knowledge. Physicians perceived patients’ reluctance to take opioids as a barrier to pain control, more so than did nurses, while nurses perceived patients’ tendency to under-report of pain as a barrier, more so than did physicians. Physicians and nurses held different perceptions regarding major cause of delay during opioid administration.

Conclusions

There were differences between physicians and nurses in knowledge and practices for cancer pain management. An effective educational strategy for cancer pain management is needed in order to improve medical professionals’ knowledge and clinical practices.     

Reliability and Validity of Quantifying Absolute Muscle Hardness Using Ultrasound Elastography

Muscle hardness is a mechanical property that represents transverse muscle stiffness. A quantitative method that uses ultrasound elastography for quantifying absolute human muscle hardness has been previously devised; however, its reliability and validity have not been completely verified. This study aimed to verify the reliability and validity of this quantitative method. The Young’s moduli of seven tissue-mimicking materials (in vitro; Young’s modulus range, 20–80 kPa; increments of 10 kPa) and the human medial gastrocnemius muscle (in vivo) were quantified using ultrasound elastography. On the basis of the strain/Young’s modulus ratio of two reference materials, one hard and one soft (Young’s moduli of 7 and 30 kPa, respectively), the Young’s moduli of the tissue-mimicking materials and medial gastrocnemius muscle were calculated. The intra- and inter-investigator reliability of the method was confirmed on the basis of acceptably low coefficient of variations (≤6.9%) and substantially high intraclass correlation coefficients (≥0.77) obtained from all measurements. The correlation coefficient between the Young’s moduli of the tissue-mimicking materials obtained using a mechanical method and ultrasound elastography was 0.996, which was equivalent to values previously obtained using magnetic resonance elastography. The Young’s moduli of the medial gastrocnemius muscle obtained using ultrasound elastography were within the range of values previously obtained using magnetic resonance elastography. The reliability and validity of the quantitative method for measuring absolute muscle hardness using ultrasound elastography were thus verified.     

Mental Health Service Use by Young People: The Role of Caregiver Characteristics

Aims

Many children and adolescents experiencing mental health problems do not receive appropriate care. Strategies to encourage appropriate access to services might be improved by a more detailed understanding of service use determinants within this group. In view of caregivers’ key role in young people’s pathways to care, this study aimed to advance understanding of caregiver-related characteristics that influence service use among young people.

Methods

We interviewed 407 primary caregivers of young people aged 9-18 years, recruited from a Greater London (United Kingdom) community sample. Caregivers reported on young people’s service use in health care sector and/or education settings, and caregivers’ intended stigmatising behaviours, help-seeking attitudes, and personal service use. Logistic regression analyses examined the relationship between these caregiver characteristics and young people’s service use, controlling for young people’s clinical and socio-demographic factors.

Results

Caregivers’ intended stigmatising behaviours in particular exerted a strong influence on young people’s service use within each service setting. The impact of this characteristic interacted with caregivers’ service use in influencing young people’s service use across health care and education settings and health care settings specifically. For young people’s service use within education settings, caregivers’ intended stigmatising behaviours score had a main effect.

Conclusions

This study highlights the key role caregivers’ attitudes and experiences hold in young people’s service use. The findings indicate that strategies aiming to bridge the gap between young people’s service needs and utilisation might be improved by targeting stigma amongst caregivers.     

Development and Psychometric Evaluation of an Instrument to Assess Cross-Cultural Competence of Healthcare Professionals (CCCHP)

Background

Cultural competence of healthcare professionals (HCPs) is recognized as a strategy to reduce cultural disparities in healthcare. However, standardised, valid and reliable instruments to assess HCPs’ cultural competence are notably lacking. The present study aims to 1) identify the core components of cultural competence from a healthcare perspective, 2) to develop a self-report instrument to assess cultural competence of HCPs and 3) to evaluate the psychometric properties of the new instrument.

Methods

The conceptual model and initial item pool, which were applied to the cross-cultural competence instrument for the healthcare profession (CCCHP), were derived from an expert survey (n = 23), interviews with HCPs (n = 12), and a broad narrative review on assessment instruments and conceptual models of cultural competence. The item pool was reduced systematically, which resulted in a 59-item instrument. A sample of 336 psychologists, in advanced psychotherapeutic training, and 409 medical students participated, in order to evaluate the construct validity and reliability of the CCCHP.

Results

Construct validity was supported by principal component analysis, which led to a 32-item six-component solution with 50% of the total variance explained. The different dimensions of HCPs’ cultural competence are: Cross-Cultural Motivation/Curiosity, Cross-Cultural Attitudes, Cross-Cultural Skills, Cross-Cultural Knowledge/Awareness and Cross-Cultural Emotions/Empathy. For the total instrument, the internal consistency reliability was .87 and the dimension’s Cronbach’s α ranged from .54 to .84. The discriminating power of the CCCHP was indicated by statistically significant mean differences in CCCHP subscale scores between predefined groups.

Conclusions

The 32-item CCCHP exhibits acceptable psychometric properties, particularly content and construct validity to examine HCPs’ cultural competence. The CCCHP with its five dimensions offers a comprehensive assessment of HCPs’ cultural competence, and has the ability to distinguish between groups that are expected to differ in cultural competence. This instrument can foster professional development through systematic self-assessment and thus contributes to improve the quality of patient care.     

Psychometric Properties of the Autoquestionnaire Qualité De Vie Enfant Imagé (AUQEI) Applied to Children with Cerebral Palsy

BackgroundQuality of life (QL) assessments of children with incapacitating diseases, such as cerebral palsy (CP), have often been conducted with the help of the representatives of a child, making QL assessment more subjective. TheAutoquestionnaireQualité de Vie Enfant Imagé (AUQEI) is a QL assessment designed for children to self-report—it uses images to facilitate the reporting process.Objectiveevaluate the psychometric properties of AUQEI when responses are given by children with CP.FindingsChildren aged 4 to 12 years (45 with CP and 45 healthy children) gave responses to the questionnaire. The data quality, reliability and validity were assessed. The data loss rate ranged from 8.8% to 46.7%, and was highest for the “autonomy” factor. No floor or ceiling effect was detected. The success rate for reliability of the internal consistency of the items was less than 80% for the “autonomy” factor. Cronbach’s alpha coefficient was 0.71 for the instrument and less than 0.5 for the factors. All the factors had a success rate of greater than 80% for the discriminating validity of the items. The factors did not have correlations between each other, thus indicating adequate discriminating validity. Convergent validity was tested and a significant correlation was demonstrated only between the AUQEI “functioning” factor and the Child Health Questionnaire—50-Item (CHQ-PF50) physical summary score (r = 0.31, p = 0.042). The AUQEI scores did not have correlations with the gross motor function scores (p>0.05) as expected for divergent validity. Regarding construct validity, the total AUQEI score obtained by the CP group was lower (median: 47.3) than that of the healthy group (median: 51.0) (p<0.01).ConclusionThe AUQEI was shown to be a reliable and valid instrument for assessing children with CP when the total score was used. Convergent validity should continue to be tested in future studies.     

Tool for Rapid & Easy Identification of High Risk Diabetic Foot: Validation & Clinical Pilot of the Simplified 60 Second Diabetic Foot Screening Tool

Background

Most diabetic foot amputations are caused by ulcers on the skin of the foot i.e. diabetic foot ulcers. Early identification of patients at high risk for diabetic foot ulcers is crucial. The ‘Simplified 60-Second Diabetic Foot Screening Tool’ has been designed to rapidly detect high risk diabetic feet, allowing for timely identification and referral of patients needing treatment. This study aimed to determine the clinical performance and inter-rater reliability of ‘Simplified 60 Second Diabetic Foot Screening Tool’ in order to evaluate its applicability for routine screening.

Methods and Findings

The tool was independently tested by n=12 assessors with n=18 Guyanese patients with diabetes. Inter-rater reliability was assessed by calculating Cronbach’s alpha for each of the assessment items. A minimum value of 0.60 was considered acceptable. Reliability scores of the screening tool assessment items were: ‘monofilament test’ 0.98; ‘active ulcer’ 0.97; ‘previous amputation’ 0.97; ‘previous ulcer’ 0.97; ‘fixed ankle’ 0.91; ‘deformity’ 0.87; ‘callus’ 0.87; ‘absent pulses’ 0.87; ‘fixed toe’ 0.80; ‘blisters’ 0.77; ‘ingrown nail’ 0.72; and ‘fissures’ 0.55. The item ‘stiffness in the toe or ankle’ was removed as it was observed in only 1.3% of patients. The item ‘fissures’ was also removed due to low inter-rater reliability. Clinical performance was assessed via a pilot study utilizing the screening tool on n=1,266 patients in an acute care setting in Georgetown, Guyana. In total, 48% of patients either had existing diabetic foot ulcers or were found to be at high risk for developing ulcers.

Conclusions

Clinicians in low and middle income countries such as Guyana can use the Simplified 60-Second Diabetic Screening Tool to facilitate early detection and appropriate treatment of diabetic foot ulcers. Implementation of this screening tool has the potential to decrease diabetes related disability and mortality.     

Interprofessional Collaboration on an Internal Medicine Ward: Role Perceptions and Expectations among Nurses and Residents

Background

Effective interprofessional collaboration requires that team members share common perceptions and expectations of each other''s roles.

Objective

Describe and compare residents’ and nurses’ perceptions and expectations of their own and each other’s professional roles in the context of an Internal Medicine ward.

Methods

A convenience sample of 14 residents and 14 nurses volunteers from the General Internal Medicine Division at the University Hospitals of Geneva, Switzerland, were interviewed to explore their perceptions and expectations of residents’ and nurses’ professional roles, for their own and the other profession. Interviews were analysed using thematic content analysis. The same respondents also filled a questionnaire asking their own intended actions and the expected actions from the other professional in response to 11 clinical scenarios.

Results

Three main themes emerged from the interviews: patient management, clinical reasoning and decision-making processes, and roles in the team. Nurses and residents shared general perceptions about patient management. However, there was a lack of shared perceptions and expectations regarding nurses’ autonomy in patient management, nurses’ participation in the decision-making process, professional interdependence, and residents’ implication in teamwork. Results from the clinical scenarios showed that nurses’ intended actions differed from residents’ expectations mainly regarding autonomy in patient management. Correlation between residents’ expectations and nurses’ intended actions was 0.56 (p = 0.08), while correlation between nurses’ expectations and residents’ intended actions was 0.80 (p<0.001).

Conclusions

There are discordant perceptions and unmet expectations among nurses and residents about each other’s roles, including several aspects related to the decision-making process. Interprofessional education should foster a shared vision of each other’s roles and clarify the boundaries of autonomy of each profession.     

Reliability and Validity of the KIPPPI: An Early Detection Tool for Psychosocial Problems in Toddlers

Background

The KIPPPI (Brief Instrument Psychological and Pedagogical Problem Inventory) is a Dutch questionnaire that measures psychosocial and pedagogical problems in 2-year olds and consists of a KIPPPI Total score, Wellbeing scale, Competence scale, and Autonomy scale. This study examined the reliability, validity, screening accuracy and clinical application of the KIPPPI.

Methods

Parents of 5959 2-year-old children in the Rotterdam area, the Netherlands, were invited to participate in the study. Parents of 3164 children (53.1% of all invited parents) completed the questionnaire. The internal consistency was evaluated and in subsamples the test-retest reliability and concurrent validity with regard to the Child Behavioral Checklist (CBCL). Discriminative validity was evaluated by comparing scores of parents who worried about their child’s upbringing and parent’s that did not. Screening accuracy of the KIPPPI was evaluated against the CBCL by calculating the Receiver Operating Characteristic (ROC) curves. The clinical application was evaluated by the relation between KIPPPI scores and the clinical decision made by the child health professionals.

Results

Psychometric properties of the KIPPPI Total score, Wellbeing scale, Competence scale and Autonomy scale were respectively: Cronbach’s alphas: 0.88, 0.86, 0.83, 0.58. Test-retest correlations: 0.80, 0.76, 0.73, 0.60. Concurrent validity was as hypothesised. The KIPPPI was able to discriminate between parents that worried about their child and parents that did not. Screening accuracy was high (>0.90) for the KIPPPI Total score and for the Wellbeing scale. The KIPPPI scale scores and clinical decision of the child health professional were related (p<0.05), indicating a good clinical application.

Conclusion

The results in this large-scale study of a diverse general population sample support the reliability, validity and clinical application of the KIPPPI Total score, Wellbeing scale and Competence scale. Also, the screening accuracy of the KIPPPI Total score and Wellbeing scale were supported. The Autonomy scale needs further study.     

Patient Experience and Satisfaction with Inpatient Service: Development of Short Form Survey Instrument Measuring the Core Aspect of Inpatient Experience

Patient experience reflects quality of care from the patients’ perspective; therefore, patients’ experiences are important data in the evaluation of the quality of health services. The development of an abbreviated, reliable and valid instrument for measuring inpatients’ experience would reflect the key aspect of inpatient care from patients’ perspective as well as facilitate quality improvement by cultivating patient engagement and allow the trends in patient satisfaction and experience to be measured regularly. The study developed a short-form inpatient instrument and tested its ability to capture a core set of inpatients’ experiences. The Hong Kong Inpatient Experience Questionnaire (HKIEQ) was established in 2010; it is an adaptation of the General Inpatient Questionnaire of the Care Quality Commission created by the Picker Institute in United Kingdom. This study used a consensus conference and a cross-sectional validation survey to create and validate a short-form of the Hong Kong Inpatient Experience Questionnaire (SF-HKIEQ). The short-form, the SF-HKIEQ, consisted of 18 items derived from the HKIEQ. The 18 items mainly covered relational aspects of care under four dimensions of the patient’s journey: hospital staff, patient care and treatment, information on leaving the hospital, and overall impression. The SF-HKIEQ had a high degree of face validity, construct validity and internal reliability. The validated SF-HKIEQ reflects the relevant core aspects of inpatients’ experience in a hospital setting. It provides a quick reference tool for quality improvement purposes and a platform that allows both healthcare staff and patients to monitor the quality of hospital care over time.     

Construct Validation of a Multidimensional Computerized Adaptive Test for Fatigue in Rheumatoid Arthritis

Objective

Multidimensional computerized adaptive testing enables precise measurements of patient-reported outcomes at an individual level across different dimensions. This study examined the construct validity of a multidimensional computerized adaptive test (CAT) for fatigue in rheumatoid arthritis (RA).

Methods

The ‘CAT Fatigue RA’ was constructed based on a previously calibrated item bank. It contains 196 items and three dimensions: ‘severity’, ‘impact’ and ‘variability’ of fatigue. The CAT was administered to 166 patients with RA. They also completed a traditional, multidimensional fatigue questionnaire (BRAF-MDQ) and the SF-36 in order to examine the CAT’s construct validity. A priori criterion for construct validity was that 75% of the correlations between the CAT dimensions and the subscales of the other questionnaires were as expected. Furthermore, comprehensive use of the item bank, measurement precision and score distribution were investigated.

Results

The a priori criterion for construct validity was supported for two of the three CAT dimensions (severity and impact but not for variability). For severity and impact, 87% of the correlations with the subscales of the well-established questionnaires were as expected but for variability, 53% of the hypothesised relations were found. Eighty-nine percent of the items were selected between one and 137 times for CAT administrations. Measurement precision was excellent for the severity and impact dimensions, with more than 90% of the CAT administrations reaching a standard error below 0.32. The variability dimension showed good measurement precision with 90% of the CAT administrations reaching a standard error below 0.44. No floor- or ceiling-effects were found for the three dimensions.

Conclusion

The CAT Fatigue RA showed good construct validity and excellent measurement precision on the dimensions severity and impact. The dimension variability had less ideal measurement characteristics, pointing to the need to recalibrate the CAT item bank with a two-dimensional model, solely consisting of severity and impact.     

Revision of the Cognitive Assessment for Dementia,iPad Version (CADi2)

Early detection of dementia is crucial because it is the time when intervention is most effective. Therefore, a simple and short test is necessary for primary mass screening in community-based medical facilities. We developed the Cognitive Assessment for Dementia, iPad version (CADi) which consists of 10 simple questions and is self-administered. In this paper we present a revised version which improves the detection of dementia. Two questions of the CADi were replaced in the latest version (CADi2). We examined the validity and reliability of the CADi2 in 27 Alzheimer’s disease patients and age-matched healthy controls. The Alzheimer’s disease patients had lower CADi2 scores and longer total response times to questions compared to the controls. The CADi2 had high sensitivity (0.85−0.96) and specificity (0.81−0.93), and showed significant correlations with existing standard neuropsychological tests. Cronbach’s alpha analysis revealed moderate consistency of the CADi2. These results support the utility of the CADi2 for primary screening for dementia.     

Development of the AGREE II,part 2: assessment of validity of items and tools to support application

Background

We established a program of research to improve the development, reporting and evaluation of practice guidelines. We assessed the construct validity of the items and user’s manual in the β version of the AGREE II.

Methods

We designed guideline excerpts reflecting high-and low-quality guideline content for 21 of the 23 items in the tool. We designed two study packages so that one low-quality and one high-quality version of each item were randomly assigned to each package. We randomly assigned 30 participants to one of the two packages. Participants reviewed and rated the guideline content according to the instructions of the user’s manual and completed a survey assessing the manual.

Results

In all cases, content designed to be of high quality was rated higher than low-quality content; in 18 of 21 cases, the differences were significant (p < 0.05). The manual was rated by participants as appropriate, easy to use, and helpful in differentiating guidelines of varying quality, with all scores above the mid-point of the seven-point scale. Considerable feedback was offered on how the items and manual of the β-AGREE II could be improved.

Interpretation

The validity of the items was established and the user’s manual was rated as highly useful by users. We used these results and those of our study presented in part 1 to modify the items and user’s manual. We recommend AGREE II (available at www.agreetrust.org) as the revised standard for guideline development, reporting and evaluation.For clinical practice guidelines to achieve their full potential as tools to assist in clinical, policy-related and system-level decisions,^1–3 they need to be of high quality and developed using rigorous methods.⁴ Thus, strategies are required to facilitate the development and reporting of guidelines and tools able to distinguish guidelines of varying quality. The AGREE Collaboration (Appraisal of Guidelines, Research and Evaluation) was the first to create a generic tool to assess the process of guideline development and reporting,^5,6 and it quickly became the standard for guideline evaluation.⁷As with any new assessment tool, ongoing development of the instrument was required to improve its measurement properties and advance the guideline enterprise. The AGREE Next Steps Consortium undertook a program of research to achieve these goals and create the next version of the tool, the AGREE II.⁸ The consortium completed two studies (parts 1 and 2). In part 1, also reported in this issue,⁹ we conducted an analysis of the performance of the new seven-point response scale, explored the usefulness of the AGREE items, and systematically identified ways in which the items and supporting document could be improved.In part 2, reported here, we aimed to test the construct validity of the items and evaluate the new supporting documentation, which was intended to facilitate efficient and accurate application of the tool.The validity of the original AGREE instrument was explored in three ways.⁵ Appraisers’ attitudes about the instrument’s usefulness and the helpfulness of the supporting documents (i.e., a user guide and training manual) were used as measures of face validity. The construct validity of the instrument was tested using three core hypotheses for each of the six domains; 3 of the possible 18 tests were supported. In retrospect, whether the hypotheses were generalizable across contexts was somewhat questionable. Finally, to establish criterion validity, correlations between users’ overall global endorsement and quality ratings of individual items were calculated. Whether global endorsements were a reasonable proxy gold standard was somewhat questionable. Further, for both the construct validity and the criterion validity, guidelines chosen in these studies were nominated by members of the research team as representing a range of quality, creating significant risks of bias.Together, these findings and methodological limitations illustrated the need for additional work to test and establish the instrument’s validity. The most fundamental concept of construct validity, in particular, had not been yet addressed —are guidelines known to be of higher quality rated more favourably using the AGREE instrument than guidelines known to be of lower quality? In addition, no study to date has tested specifically whether the instructions for applying the tool are perceived to be appropriate, implementable, and helpful in differentiating among guidelines of varying quality. These perceptions are important components that contribute to the face validity of the tool.We tested two specific research questions in this study. First, do the items in β-AGREE II differentiate between guideline content of known, varying quality? Second, is the new user’s manual perceived by users as appropriate, easy to apply and helpful in differentiating good quality guidelines from poor quality guidelines?     

What Would Jaws Do? The Tyranny of Film and the Relationship between Gaze and Higher-Level Narrative Film Comprehension

What is the relationship between film viewers’ eye movements and their film comprehension? Typical Hollywood movies induce strong attentional synchrony—most viewers look at the same things at the same time. Thus, we asked whether film viewers’ eye movements would differ based on their understanding—the mental model hypothesis—or whether any such differences would be overwhelmed by viewers’ attentional synchrony—the tyranny of film hypothesis. To investigate this question, we manipulated the presence/absence of prior film context and measured resulting differences in film comprehension and eye movements. Viewers watched a 12-second James Bond movie clip, ending just as a critical predictive inference should be drawn that Bond’s nemesis, “Jaws,” would fall from the sky onto a circus tent. The No-context condition saw only the 12-second clip, but the Context condition also saw the preceding 2.5 minutes of the movie before seeing the critical 12-second portion. Importantly, the Context condition viewers were more likely to draw the critical inference and were more likely to perceive coherence across the entire 6 shot sequence (as shown by event segmentation), indicating greater comprehension. Viewers’ eye movements showed strong attentional synchrony in both conditions as compared to a chance level baseline, but smaller differences between conditions. Specifically, the Context condition viewers showed slightly, but significantly, greater attentional synchrony and lower cognitive load (as shown by fixation probability) during the critical first circus tent shot. Thus, overall, the results were more consistent with the tyranny of film hypothesis than the mental model hypothesis. These results suggest the need for a theory that encompasses processes from the perception to the comprehension of film.