Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review

Background

Recent reviews have synthesised the psychometric properties of measures developed to examine implementation science constructs in healthcare and mental health settings. However, no reviews have focussed primarily on the properties of measures developed to assess innovations in public health and community settings. This review identified quantitative measures developed in public health and community settings, examined their psychometric properties, and described how the domains of each measure align with the five domains and 37 constructs of the Consolidated Framework for Implementation Research (CFIR).

Methods

MEDLINE, PsycINFO, EMBASE, and CINAHL were searched to identify publications describing the development of measures to assess implementation science constructs in public health and community settings. The psychometric properties of each measure were assessed against recommended criteria for validity (face/content, construct, criterion), reliability (internal consistency, test-retest), responsiveness, acceptability, feasibility, and revalidation and cross-cultural adaptation. Relevant domains were mapped against implementation constructs defined by the CFIR.

Results

Fifty-one measures met the inclusion criteria. The majority of these were developed in schools, universities, or colleges and other workplaces or organisations. Overall, most measures did not adequately assess or report psychometric properties. Forty-six percent of measures using exploratory factor analysis reported >50 % of variance was explained by the final model; none of the measures assessed using confirmatory factor analysis reported root mean square error of approximation (<0.06) or comparative fit index (>0.95). Fifty percent of measures reported Cronbach’s alpha of <0.70 for at least one domain; 6 % adequately assessed test-retest reliability; 16 % of measures adequately assessed criterion validity (i.e. known-groups); 2 % adequately assessed convergent validity (r?>?0.40). Twenty-five percent of measures reported revalidation or cross-cultural validation. The CFIR constructs most frequently assessed by the included measures were relative advantage, available resources, knowledge and beliefs, complexity, implementation climate, and other personal resources (assessed by more than ten measures). Five CFIR constructs were not addressed by any measure.

Conclusions

This review highlights gaps in the range of implementation constructs that are assessed by existing measures developed for use in public health and community settings. Moreover, measures with robust psychometric properties are lacking. Without rigorous tools, the factors associated with the successful implementation of innovations in these settings will remain unknown

     

Light Intensity Physical Activity and Sedentary Behavior in Relation to Body Mass Index and Grip Strength in Older Adults: Cross-Sectional Findings from the Lifestyle Interventions and Independence for Elders (LIFE) Study

BackgroundIdentifying modifiable determinants of fat mass and muscle strength in older adults is important given their impact on physical functioning and health. Light intensity physical activity and sedentary behavior are potential determinants, but their relations to these outcomes are poorly understood. We evaluated associations of light intensity physical activity and sedentary time—assessed both objectively and by self-report—with body mass index (BMI) and grip strength in a large sample of older adults.MethodsWe used cross-sectional baseline data from 1130 participants of the Lifestyle Interventions and Independence for Elders (LIFE) study, a community-dwelling sample of relatively sedentary older adults (70-89 years) at heightened risk of mobility disability. Time spent sedentary and in light intensity activity were assessed using an accelerometer worn for 3–7 days (Actigraph GT3X) and by self-report. Associations between these exposures and measured BMI and grip strength were evaluated using linear regression.ResultsGreater time spent in light intensity activity and lower sedentary times were both associated with lower BMI. This was evident using objective measures of lower-light intensity, and both objective and self-reported measures of higher-light intensity activity. Time spent watching television was positively associated with BMI, while reading and computer use were not. Greater time spent in higher but not lower intensities of light activity (assessed objectively) was associated with greater grip strength in men but not women, while neither objectively assessed nor self-reported sedentary time was associated with grip strength.ConclusionsIn this cross-sectional study, greater time spent in light intensity activity and lower sedentary times were associated with lower BMI. These results are consistent with the hypothesis that replacing sedentary activities with light intensity activities could lead to lower BMI levels and obesity prevalence among the population of older adults. However, longitudinal and experimental studies are needed to strengthen causal inferences.     

A Delphi Technology Foresight Study: Mapping Social Construction of Scientific Evidence on Metagenomics Tests for Water Safety

Access to clean water is a grand challenge in the 21^st century. Water safety testing for pathogens currently depends on surrogate measures such as fecal indicator bacteria (e.g., E. coli). Metagenomics concerns high-throughput, culture-independent, unbiased shotgun sequencing of DNA from environmental samples that might transform water safety by detecting waterborne pathogens directly instead of their surrogates. Yet emerging innovations such as metagenomics are often fiercely contested. Innovations are subject to shaping/construction not only by technology but also social systems/values in which they are embedded, such as experts’ attitudes towards new scientific evidence. We conducted a classic three-round Delphi survey, comprised of 107 questions. A multidisciplinary expert panel (n = 24) representing the continuum of discovery scientists and policymakers evaluated the emergence of metagenomics tests. To the best of our knowledge, we report here the first Delphi foresight study of experts’ attitudes on (1) the top 10 priority evidentiary criteria for adoption of metagenomics tests for water safety, (2) the specific issues critical to governance of metagenomics innovation trajectory where there is consensus or dissensus among experts, (3) the anticipated time lapse from discovery to practice of metagenomics tests, and (4) the role and timing of public engagement in development of metagenomics tests. The ability of a test to distinguish between harmful and benign waterborne organisms, analytical/clinical sensitivity, and reproducibility were the top three evidentiary criteria for adoption of metagenomics. Experts agree that metagenomic testing will provide novel information but there is dissensus on whether metagenomics will replace the current water safety testing methods or impact the public health end points (e.g., reduction in boil water advisories). Interestingly, experts view the publics relevant in a “downstream capacity” for adoption of metagenomics rather than a co-productionist role at the “upstream” scientific design stage of metagenomics tests. In summary, these findings offer strategic foresight to govern metagenomics innovations symmetrically: by identifying areas where acceleration (e.g., consensus areas) and deceleration/reconsideration (e.g., dissensus areas) of the innovation trajectory might be warranted. Additionally, we show how scientific evidence is subject to potential social construction by experts’ value systems and the need for greater upstream public engagement on metagenomics innovations.     

Evidence for a Cystic Fibrosis Enteropathy

BackgroundPrevious studies have suggested the existence of enteropathy in cystic fibrosis (CF), which may contribute to intestinal function impairment, a poor nutritional status and decline in lung function. This study evaluated enterocyte damage and intestinal inflammation in CF and studied its associations with nutritional status, CF-related morbidities such as impaired lung function and diabetes, and medication use.MethodsSixty-eight CF patients and 107 controls were studied. Levels of serum intestinal-fatty acid binding protein (I-FABP), a specific marker for enterocyte damage, were retrospectively determined. The faecal intestinal inflammation marker calprotectin was prospectively studied. Nutritional status, lung function (FEV1), exocrine pancreatic insufficiency (EPI), CF-related diabetes (CFRD) and use of proton pump inhibitors (PPI) were obtained from the medical charts.ResultsSerum I-FABP levels were elevated in CF patients as compared with controls (p<0.001), and correlated negatively with FEV1 predicted value in children (r-.734, p<0.05). Faecal calprotectin level was elevated in 93% of CF patients, and correlated negatively with FEV1 predicted value in adults (r-.484, p<0.05). No correlation was found between calprotectin levels in faeces and sputum. Faecal calprotectin level was significantly associated with the presence of CFRD, EPI, and PPI use.ConclusionThis study demonstrated enterocyte damage and intestinal inflammation in CF patients, and provides evidence for an inverse correlation between enteropathy and lung function. The presented associations of enteropathy with important CF-related morbidities further emphasize the clinical relevance.     

A Statistical Evaluation of Sediment Quality Triad Benthic Community Measurements in PAH-Impacted Sediments at a Manufactured Gas Plant Site along the Hudson River

The sediment quality triad (SQT) assumes that three measurements (sediment chemistry, laboratory bioassay, and benthic macroinvertebrate counts) comprise an independent assessment of impact, which when integrated using a weight-of-evidence approach provides a comprehensive assessment of risk. An SQT assessment was conducted on 41 sediment samples collected adjacent to a manufactured gas plant site on the freshwater reach of the Hudson River in New York State. The assessment shows that the benthic macroinvertebrate data did not correlate with either sediment or pore water polycyclic aromatic hydrocarbon (PAH) concentrations, nor did these data show consistent relationships to the results of laboratory bioassay testing (Hyalella azteca 28-day survival or biomass). The benthic community across the site and reference areas was comprised of few taxa, all of which were pollution-tolerant organisms with tolerance values greater than or equal to five. Only in significantly impacted sediment samples with PAH concentrations in the thousands of milligrams per kilogram, pore water concentrations above 100 toxic units, and visible non-aqueous phase liquid present in the sample did the benthic macroinvertebrate data show a response. In contrast, sediment and pore water PAH measurements and H. azteca toxicity testing provided consistent interpretation of impact. These results illustrate that benthic macroinvertebrate data may contain less information value and be a more challenging line of evidence to interpret in triad studies conducted in certain ecological settings; in this case, a large-order river with a relatively depauperate benthic community dominated by species tolerant of PAHs.     

Inferring the Forces Controlling Metaphase Kinetochore Oscillations by Reverse Engineering System Dynamics

Kinetochores are multi-protein complexes that mediate the physical coupling of sister chromatids to spindle microtubule bundles (called kinetochore (K)-fibres) from respective poles. These kinetochore-attached K-fibres generate pushing and pulling forces, which combine with polar ejection forces (PEF) and elastic inter-sister chromatin to govern chromosome movements. Classic experiments in meiotic cells using calibrated micro-needles measured an approximate stall force for a chromosome, but methods that allow the systematic determination of forces acting on a kinetochore in living cells are lacking. Here we report the development of mathematical models that can be fitted (reverse engineered) to high-resolution kinetochore tracking data, thereby estimating the model parameters and allowing us to indirectly compute the (relative) force components (K-fibre, spring force and PEF) acting on individual sister kinetochores in vivo. We applied our methodology to thousands of human kinetochore pair trajectories and report distinct signatures in temporal force profiles during directional switches. We found the K-fibre force to be the dominant force throughout oscillations, and the centromeric spring the smallest although it has the strongest directional switching signature. There is also structure throughout the metaphase plate, with a steeper PEF potential well towards the periphery and a concomitant reduction in plate thickness and oscillation amplitude. This data driven reverse engineering approach is sufficiently flexible to allow fitting of more complex mechanistic models; mathematical models of kinetochore dynamics can therefore be thoroughly tested on experimental data for the first time. Future work will now be able to map out how individual proteins contribute to kinetochore-based force generation and sensing.