Multiple observers, humidity, and choice of precision statistics: factors influencing craniometric data quality

This study investigates three topics: (1) interobserver measurement error in craniometry, (2) the effects of humidity on craniometric measurements, and (3) the current status of estimators of measurement precision in craniometry and anthropometry. The results of the three-observer error analysis based on 24 linear measurements taken on 47 crania indicate that minor idiosyncratic variations in measurement technique can lead to high levels of statistical discrimination among the data produced by the different observers. The results of the humidity experiment substantiate the contention that increasing levels of relative humidity are associated with cranial expansion. The results of the comparison of 11 univariate precision estimators suggest that the combination of percentage agreement, the mean absolute difference, and Fisher's nonparametric sign test can give an instructive picture of the frequency, magnitude, and directionality of measurement imprecision. Information on the comparability of technique and measurement precision can then be used in the variable selection process prior to the application of multivariate statistical procedures to strengthen the substantive interpretation of craniometric data.     

How long is a piece of Strix? Methods in measuring and measuring the measurers

An experiment to quantify intra- and interobserver error in anatomical measurements found that interobserver measurements can vary by over 14% of mean specimen length; disparity in measurement increases logarithmically with the number of contributors; instructions did not reduce variation or measurement disparity; scale of the specimen influenced the precision of measurement (relative error increasing with specimen size); different methods of taking a measurement yielded different results, although they did not differ in terms of precision, and topographical complexity of the elements being considered may potentially influence error (error increasing with complexity). These results highlight concerns about introduction of noise and potential bias that should be taken into account when compiling composite datasets and meta-analyses.     

Technical note: Comparability of Hrdlička's Catalog of crania data based on measurement landmark definitions

Archival sources of data are critical anthropological resources that inform inferences about human biology and evolutionary history. Craniometric data are one of the most widely available sources of information on human population history because craniometrics were critical in early 20th century debates about race and biological variation. As such, extensive databases of raw craniometric data were published at the same time that the field was working to standardize measurement protocol. Hrdli?ka published between 10 and 16 raw craniometric variables for over 8,000 individuals in a series of seven catalogs throughout his career. With a New World emphasis, Hrdli?ka's data complement those of Howells ( 1973 , 1989 ) and the two databases have been combined in the past. In this note we verify the consistency of Hrdli?ka's measurement protocol throughout the Catalog series and compare these definitions to those used by Howells. We conclude that 12 measurements are comparable throughout the Catalogs, with five of these equivalent to Howells' measurements: maximum cranial breadth (XCB), basion‐bregma height (BBH), maximum bizygomatic breadth (ZYB), nasal breadth (NLB), and breadth of the upper alveolar arch (MAB). Most of Hrdli?ka's measurements are not strictly comparable to those of Howells, thus limiting the utility of combined datasets for multivariate analysis. Four measurements are inconsistently defined by Hrdli?ka and we recommend not using these data: nasal height, orbit breadth, orbit height, and menton‐nasion height. This note promotes Hrdli?ka's tireless efforts at data collection and re‐emphasizes observer error as a legitimate concern in craniometry as the field shifts to morphometric digital data acquisition. Am J Phys Anthropol, 2011. © 2011 Wiley‐Liss, Inc.     

Interobserver errors in anthropometry

To present basic information on the interobserver precision and accuracy of 32 selected anthropometric measurement items, six observers measured each of 37 subjects once in two days. The data were analyzed by using ANOVA, and mean absolute bias, standard deviation of bias, and mean absolute bias in standard deviation unit were used as measures of bias. By comparing the results of the two days, the effects of the practice on measurement errors were also investigated. Variance was overestimated by more than 10% in five measurements. Interobserver error variance and random error variance were highly correlated with each other. Measures of the bias were significantly correlated with interobserver and especially with random error variances. The interobserver errors were drastically reduced on the second day in the measurement items in which the causes of the interobserver errors could be specified. It was speculated that even when the definitions of the landmarks and measurement items were clear, the ambiguity in the practical procedures in locating landmarks, applying instruments, and so on, permitted each observer to develop his or her own measurement technique, and it in turn caused interobserver errors. To minimize interobserver and random errors, the standardization of measurement technique should be extended to the details of the practical procedures.     

Comparison of 2.5D and 3D Quantification of Femoral Head Coverage in Normal Control Subjects and Patients with Hip Dysplasia

Hip dysplasia is characterized by insufficient femoral head coverage (FHC). Quantification of FHC is of importance as the underlying goal of the surgery to treat hip dysplasia is to restore a normal acetabular morphology and thereby to improve FHC. Unlike a pure 2D X-ray radiograph-based measurement method or a pure 3D CT-based measurement method, previously we presented a 2.5D method to quantify FHC from a single anteriorposterior (AP) pelvic radiograph. In this study, we first quantified and compared 3D FHC between a normal control group and a patient group using a CT-based measurement method. Taking the CT-based 3D measurements of FHC as the gold standard, we further quantified the bias, precision and correlation between the 2.5D measurements and the 3D measurements on both the control group and the patient group. Based on digitally reconstructed radiographs (DRRs), we investigated the influence of the pelvic tilt on the 2.5D measurements of FHC. The intraclass correlation coefficients (ICCs) for absolute agreement was used to quantify interobserver reliability and intraobserver reproducibility of the 2.5D measurement technique. The Pearson correlation coefficient, r, was used to determine the strength of the linear association between the 2.5D and the 3D measurements. Student’s t-test was used to determine whether the differences between different measurements were statistically significant. Our experimental results demonstrated that both the interobserver reliability and the intraobserver reproducibility of the 2.5D measurement technique were very good (ICCs > 0.8). Regression analysis indicated that the correlation was very strong between the 2.5D and the 3D measurements (r = 0.89, p < 0.001). Student’s t-test showed that there were no statistically significant differences between the 2.5D and the 3D measurements of FHC on the patient group (p > 0.05). The results of this study provided convincing evidence demonstrating the validity of the 2.5D measurements of FHC from a single AP pelvic radiograph and proved that it could serve as a surrogate for 3D CT-based measurements. Thus it may be possible to use this method to avoid a CT scan for the purpose of estimating 3D FHC in diagnosis and post-operative treatment evaluation of patients with hip dysplasia.     

运用CT三维重建测量重度先天性脊柱侧凸Cobb角的临床应用

目的:评价在CT三维重建上测量重度先天性脊柱侧凸Cobb角度的可重复性和可靠性。方法:收集在我院诊治的重度先天性脊柱侧凸病人的CT三维重建和脊柱全长X线片,共计67例。由五名不同测量者对CT三维重建脊柱畸形冠状面主弯Cobb角测量两次,两次间隔在三周以上,并测量脊柱全长X线片脊柱畸形冠状面主弯Cobb角一次,运用组内相关系数分析测量结果之间的可重复性和可靠性。结果:同一测量者两次测量结果之间的差值平均为4.5°。同一测量者两次测量结果之间的组内相关系数为0.969,不同测量者之间测量结果的组内相关系数为0.913。取五名测量者在CT三维重建上第一次测量的Cobb角度,其平均值为(110.5±23.5)°,五名测量者在X线片上测量结果的平均值为(103.1±22.0)°,对两组数据进行Mann-Whitney非参数检验差异有统计学意义(Z=-2.86,P=0.004)。结论:在CT三维重建上测量重度先天性脊柱侧凸的Cobb角,可以减小测量的误差,提高测量的可重复性和可靠性,是一种相对准确的测量方法。     

Technical note: estimating sex using cervical canine odontometrics: a test using a known sex sample

The size of the permanent human canine tooth is one of the few sexually dimorphic features to be present in childhood and as such offers the opportunity to assist in the identification of sex in remains where no other appropriate criteria exist, such as in subadults. However, canine odontometrics are often associated with high levels of interobserver error and can be difficult to access if dentition is in situ. Additionally, appropriate points of measurement can be difficult to identify if the tooth is worn. Alternate measurements of the cervical canine diameters have been proposed as solutions to these issues, but the utility of these measurements in estimating sex has not been conclusively demonstrated. This study uses the buccolingual and mesiodistal cervical diameter of the canines from a known-sex sample from St. Bride's Church, London and a partially known-sex sample from the Old Church, Chelsea, London to classify individuals as male or female. A discriminant function classification using these diameters successfully identifies sex in 93.8% of the known-sex assemblage and 95% of the partially osteologically estimated sex assemblage. It is suggested that cervical canine diameters are highly repeatable measurements with low interobserver error, can be obtained on worn and in situ teeth, and provide as good or better guidance on estimating sex in human remains as standard maximal diameters.     

Measurement standards for human metacarpals

Standards for measuring the metacarpals are absent from commonly used osteometric guides. Perhaps the closest to a set of standard measurements in common use today are those proposed by Scheuer and Elkington (Scheuer and Elkington: J Forensic Sci 38 (1993) 769–788) for forensic sex assessment. They include caliper measurements of interarticular length, base and head width, base and head height, and maximum midshaft diameter. Over the last decade, a new set of measurements that encompass similar dimensions to those used by Scheuer and Elkington, but which are taken with a mini‐osteometric board (MOB) have been developed by the lead author. Use of the MOB avoids the need to manipulate both the bone and calipers in three‐dimensional space and causes less strain on the hands. However, the question of intra‐ and interobserver accuracy has not been adequately addressed for either set of measurements. The purpose of this study was to test both the Scheuer/Elkington and MOB measurements on 20 hands from 10 anatomical skeletons for intra‐ and inter‐observer accuracy. The study found that 92% of the MOB measures had a lower intraobserver error, and 88% had a lower interobserver error than did the caliper measurements. It also found that the maximum midshaft diameter measurement used by Scheuer and Elkington is more repeatable than a mediolateral diameter. Overall, 88% of the 25 MOB measurements had median intraobserver error rates of under 1.5%, compared with 60% of the caliper measurements. Furthermore, the MOB measurements as a set were taken 10 to 12% faster than the caliper measurements. Am J Phys Anthropol 157:322–329, 2015. © 2015 Wiley Periodicals, Inc.     

Stability of hemoglobin mass over 100 days in active men

The purpose of this study was to investigate the suggestion in a recent meta-analysis that variability in hemoglobin mass increases when time between measurements increases from days to months. Hemoglobin mass of six active men was measured with the carbon monoxide method every 1-6 days for 100-114 days (42 +/- 3 measurements, mean +/- SD). Measurement error for each individual's series was estimated from the standard deviation of consecutive pairwise changes and compared with his total error (standard deviation of all values). Linear trends and periodicities in each series were quantified by regression and spectral analysis. Series with known random error and periodicity were also simulated and analyzed. There were clear differences in the pairwise error of measurement between subjects (range 1.4-2.7%). For five men, there was little difference between the total and pairwise errors; their mean ratio (1.06, 90% confidence limits 0.96-1.17) was less than ratios for simulated sinusoidal series with random error of 2%, amplitude of 2%, and periods of 20-100 days (ratios 1.13-1.21). Spectral analysis clearly revealed such periodicities in the simulated series but not in the series of these subjects. The sixth man, who had donated blood 12 days before commencing measurements, showed errors, trend, and periodicity consistent with gradual restoration of hemoglobin mass. Measurement error of hemoglobin mass does not increase over 100 days. Consequently, hemoglobin mass may be suitable for long-term monitoring of small changes that might occur with training or erythropoietin abuse, taking into consideration the small differences between athletes in errors and trends.     

Optimal selection of metabolic fluxes for in vivo measurement. II. Application to Escherichia coli and hybridoma cell metabolism.

A method of analysis was presented in part I of this series for determining the fluxes in a biochemical network that are the optimal choices for experimental measurement. This algorithm is applied to two important biological models: Escherichia coli and a hybridoma cell line (167.4G5.3). Our results show that potentially poor choices for in vivo measurement of metabolic fluxes exist for both model systems. For the subset of reactions in E. coli that was studied, the condition number of the augmented stoichiometric matrix reveals that a 60-fold amplification of experimental error during computations is possible. The biochemical network of the hybridoma cell is more complex than the E. coli system, and thus results in much larger possible error amplification--up to 100,000-fold. The physiological situations appear to have sensitivities that are less than 1/4 to 1/10 of those estimated by the condition number, and the maximum sensitivities are proportional to the condition number. These maximum sensitivities calculated using estimates of the fluxes and the worst possible error vector are upper bounds on the system's actual sensitivity. By examining the effect of measurement error on the sensitivity, the most probable sensitivity is calculated. These results indicate that an approximate two-fold increase in sensitivity of the E. coli system is likely when the worst set of fluxes are measured rather than the best set. The most likely sensitivity of the hybridoma system can range three orders of magnitude, depending on the set of fluxes that are measured. The propagation of experimental error during computations can be diminished for both systems by increasing the number of flux measurements over and above the minimum number of experimental measurements. The findings from these two model systems indicate that the calculation of the condition number can be a useful method for efficient experimental design, and that the usefulness of this method increases as the order of the system increases.     

A note on interobserver error in multivariate analyses of populations

The effect of interobserver error on a principal components analysis of a small sample of human crania is examined. A comparison of individual specimen scores for components is made to find rotated principal components which identify interobserver error. The individual variables which load highly on such components are then tested for interobserver error univariately. Multivariate components which must identify interobserver error contain no high loadings for variables which demonstrate interobserver error in the univariate case. Principal component analysis, in defining new component variables, extracts such error in an easily identified way which makes comparison of samples measured by more than one anthropometrist more reliable.     

Sexual dimorphism in human skulls. A comparison of sexual dimorphism in different populations

Application and comparison of sex discriminant functions in different populations led to the conclusion that a certain combination and weighting of a few sex dimorphism variables (in this study we only used craniometric variables) can give a good discrimination between male and female individuals, independent of the racial group to which this function is applied. In our study, the sex-discriminatory power of five discriminant functions which were based on different ordination and selection procedures (e.g. professional knowledge, stepwise discriminant analysis, literature) of the cranial variables is compared. These discriminant functions were applied to three different data sets, the first being skull measurements from an Amsterdam series (Europids), the second skull measurements of a Zulu series (Negrids) and the third skull measurements of a Japan series (Mongolids). Our decision as to whether a function is a good or less good sex-discriminating function is determined by the Dt values (these values give an idea about the discriminatory value of the discriminant function when applied to a new test sample), the number of variables necessary to obtain this Dt and the location of the sectioning point (i.e. comparison between the estimation of the sectioning point and the ”real” sectioning point). These discriminant functions were compared withGiles Elliot's (1962, 1963) “race-independent” sex function.     

Antenatal screening by measurement of symphysis-fundus height.

A study was undertaken to assess the value of symphysis-fundus measurement as a screening procedure for intrauterine growth retardation. The reproducibility of this measurement was investigated in two groups of six patients, each measured six times by six different observers. The intraobserver coefficient of variation was 4.6% and the interobserver coefficient of variation 6.4%. There was no evidence that experience aided consistency. A chart of symphysis-fundus measurements derived from Cardiff data was found to be similar to others previously published, and one measurement below the 10th centile identified 64% of pregnancies in which the eventual birth weight was below the 10th centile for gestational age. Symphysis-fundus measurement is a useful screening test; one chart could be used for any Caucasian population and should be incorporated into the maternity services "co-operation card."     

Missing data in craniometrics: a simulation study

Craniometric measurements represent a useful tool for studying the differentiation of mammal populations. However, the fragility of skulls often leads to incomplete data matrices. Damaged specimens or incomplete sets of measurements are usually discarded prior to statistical analysis. We assessed the performance of two strategies that avoid elimination of observations: (1) pairwise deletion of missing cells, and (2) estimation of missing data using available measurements. The effect of these distinct approaches on the computation of inter-individual distances and population differentiation analyses were evaluated using craniometric measurements obtained from insular populations of deer micePeromyscus maniculatus (Wagner, 1845). In our simulations, Euclidean distances were greatly altered by pairwise deletion, whereas Gower’s distance coefficient corrected for missing data provided accurate results. Among the different estimation methods compared in this paper, the regression-based approximations weighted by coefficients of determination (r ²) outperformed the competing approaches. We further show that incomplete sets of craniometric measurements can be used to compute distance matrices, provided that an appropriate coefficient is selected. However, the application of estimation procedures provides a flexible approach that allows researchers to analyse incomplete data sets.     

A model-based approach to measure the minimum joint space width of total knee replacements in standard radiographs

Excessive wear in total knee arthroplasty is detected by measuring the minimum joint space width (mJSW) in anterioposterior radiographs. The accuracy of conventional measurement methods is limited and can be improved using model-based techniques. In this study, the model-based wear measurement (MBWM) is introduced. Its accuracy and reproducibility are assessed and compared to the conventional measurement. Forty anterioposterior radiographs were obtained of a knee prosthesis using a phantom setup. Both measurement methods were applied and the accuracy and precision were compared. The reproducibility was calculated with inter- and intra-observer experiments. Three observers measured the mJSW in 30 clinical radiographs with both the conventional measurement and the MBWM and repeated this after 6 weeks. The experiments were conducted with a NexGen mobile bearing and fixed bearing prostheses. In the phantom experiment, the accuracy (mean of the absolute error) was significantly higher (t-test, p<0.01) for the MBWM as for the conventional measurement (0.15 mm versus 0.43 mm, 0.14 mm versus 0.35 mm for the mobile and fixed bearing, respectively). The standard deviation of the measurements is the smallest for the MBWM measurement for both prosthesis types (0.16 mm versus 0.47 mm, Levene's test, p<0.01). In the reproducibility experiment, both the intra- and inter-observer agreements were higher for the MBWM than for the conventional method. The results show that the MBWM is superior to the conventional measurement in both accuracy and reproducibility. Although the use of a phantom experiment poses some limitations in conveying the findings to clinical practice, this improved mJSW measurement can lead to better wear detection for surgery decisions and research purposes.     

Mixtures of Berkson and classical covariate measurement error in the linear mixed model: Bias analysis and application to a study on ultrafine particles

The ultrafine particle measurements in the Augsburger Umweltstudie, a panel study conducted in Augsburg, Germany, exhibit measurement error from various sources. Measurements of mobile devices show classical possibly individual–specific measurement error; Berkson–type error, which may also vary individually, occurs, if measurements of fixed monitoring stations are used. The combination of fixed site and individual exposure measurements results in a mixture of the two error types. We extended existing bias analysis approaches to linear mixed models with a complex error structure including individual–specific error components, autocorrelated errors, and a mixture of classical and Berkson error. Theoretical considerations and simulation results show, that autocorrelation may severely change the attenuation of the effect estimations. Furthermore, unbalanced designs and the inclusion of confounding variables influence the degree of attenuation. Bias correction with the method of moments using data with mixture measurement error partially yielded better results compared to the usage of incomplete data with classical error. Confidence intervals (CIs) based on the delta method achieved better coverage probabilities than those based on Bootstrap samples. Moreover, we present the application of these new methods to heart rate measurements within the Augsburger Umweltstudie: the corrected effect estimates were slightly higher than their naive equivalents. The substantial measurement error of ultrafine particle measurements has little impact on the results. The developed methodology is generally applicable to longitudinal data with measurement error.     

Estimation of body weights from craniometric variables in baboons

Body weights of adult baboons (genera Papio, Mandrillus, and Theropithecus) were gathered from notes of collectors and museum records. However, these data were insufficient to establish mean body weights for all baboon groups. Thus, log cube roots of mean body weights were regressed as functions of the logs of several cranial and dental variables. The resulting least squares regression coefficients were used to estimate weights for 503 adult baboons from cranial measurements. The ability of the various regression functions to assess baboon body weight was determined by comparing reported and estimated mean and individual body weights. The best estimator of baboon body weights was the function derived from the factor scores of a principal components analysis of seven craniometric variables regressed on body weight. However, each of these craniometric variables singly was nearly as precise an estimator of body weight as the multivariate combination of all seven. Other measurements such as dental dimensions and foramen magnum area estimated weight less accurately. Body weight estimates derived from the regression analyses coupled with museum and literature records allowed an assessment of size relationships among all baboon groups.     

Optimal selection of metabolic fluxes for in vivo measurement. I. Development of mathematical methods.

The measurement of uptake and secretion rates is often not sufficient to allow the calculation of all internal metabolic fluxes. Measurements of internal fluxes are needed and these additional measurements are used in conjunction with mass-balance equations to calculate the complete metabolic flux map. A method is presented that identifies the fluxes that should be selected for experimental measurement, and the fluxes that can be computed using the mass-balance equations. The criterion for selecting internal metabolic fluxes for measurement is that the values of the computed fluxes should have low sensitivity to experimental error in the measured fluxes. A condition number indicating the upper bound on this sensitivity, is calculated based on stoichiometry alone. The actual sensitivity is dependent on both the flux measurements and the error in flux measurements, as well as the stoichiometry. If approximate physiologic ranges of fluxes are known a realistic sensitivity can be computed. The exact sensitivity cannot be calculated since the experimental error is usually unknown. The most probable value of the actual sensitivity for a given selection of measured fluxes is estimated by selecting a large number of representative error vectors and calculating the actual sensitivity for each of these. A frequency distribution of actual sensitivities is thus obtained giving a representative range of actual sensitivities for a particular experimental situation.     

A digital registration system for net photosynthesis and transpiration measurements in the field and an associated analysis of errors

Summary A digital registration system used with temperature- and humidity-controlled cuvettes for net photosynthesis and transpiration measurements in the field is described. The associated errors of the measured parameters and calculated data are estimated. The digitalization is based on an analogue registration which is of primary importance in the control of experimental conditions in the cuvettes. The digital system is connected to the analogue registration in series. The error associated with digitalization is 0.1% across 70% of the scale. This error increases to 0.2% between 3 and 30% on the scale due to a minor lack of linearity. The reproducibility of the digitalization is ±0.024%.The error associated with data transfer in the digitalization and the errors of the analogue registration are estimated for temperature and humidity measurements (error of air and leaf temperature is ±0.1° C; error of the dew point temperature is ±1.1° C dew point). The effect of these errors on the calculation of relative humidity and the water vapour difference between the leaf and the air is determined using the progressive error law. At 30° C and 50% relative humidity, the error in relative humidity is ±7.4%, the error for the water vapour difference is ±6.6%. The dependence of these errors on temperature and humidity is shown.The instrument error of the net photosynthesis measurement is calculated to be ±4.2%. Transpiration measurements have an average inaccuracy of ±8.3%. The total diffusion resistance which is calculated from values of transpiration and the water vapour difference has an average error of ±10.9%. The sizeable influence of errors in humidity and temperature measurements on the calculated diffusion resistance is demonstrated. The additional influence of biological errors associated with field measurements is discussed.     

A measurement error model for time-series studies of air pollution and mortality

One barrier to interpreting the observational evidence concerning the adverse health effects of air pollution for public policy purposes is the measurement error inherent in estimates of exposure based on ambient pollutant monitors. Exposure assessment studies have shown that data from monitors at central sites may not adequately represent personal exposure. Thus, the exposure error resulting from using centrally measured data as a surrogate for personal exposure can potentially lead to a bias in estimates of the health effects of air pollution. This paper develops a multi-stage Poisson regression model for evaluating the effects of exposure measurement error on estimates of effects of particulate air pollution on mortality in time-series studies. To implement the model, we have used five validation data sets on personal exposure to PM10. Our goal is to combine data on the associations between ambient concentrations of particulate matter and mortality for a specific location, with the validation data on the association between ambient and personal concentrations of particulate matter at the locations where data have been collected. We use these data in a model to estimate the relative risk of mortality associated with estimated personal-exposure concentrations and make a comparison with the risk of mortality estimated with measurements of ambient concentration alone. We apply this method to data comprising daily mortality counts, ambient concentrations of PM10measured at a central site, and temperature for Baltimore, Maryland from 1987 to 1994. We have selected our home city of Baltimore to illustrate the method; the measurement error correction model is general and can be applied to other appropriate locations.Our approach uses a combination of: (1) a generalized additive model with log link and Poisson error for the mortality-personal-exposure association; (2) a multi-stage linear model to estimate the variability across the five validation data sets in the personal-ambient-exposure association; (3) data augmentation methods to address the uncertainty resulting from the missing personal exposure time series in Baltimore. In the Poisson regression model, we account for smooth seasonal and annual trends in mortality using smoothing splines. Taking into account the heterogeneity across locations in the personal-ambient-exposure relationship, we quantify the degree to which the exposure measurement error biases the results toward the null hypothesis of no effect, and estimate the loss of precision in the estimated health effects due to indirectly estimating personal exposures from ambient measurements.