Testing for Validity and Reliability in Cross-Cultural Research

A set of procedures is offered for assessing interraler reliability and certain aspects of validity of codes in cross-cultural studies. The method assumes that at least two independent raters have coded more than one trait. Each trait coded by one rater is correlated with each trait coded by a second, and all the codings by a single rater are intercorrelated with each other. The results are presented in a multitrait-multi-rater matrix. From this matrix it is possible to determine the interrater reliability and discriminant validity of trails in addition to a higher order concept based on pairs of traits.