Psychometric equivalence.

The research instruments must also be developed in a manner that ensures measurement equivalence. A fundamental issue in evaluating psychometric equivalence is the degree to which the instruments used in the cultures participating in the research measure the same construct. If equivalence is attained the researcher should expect the same or similar order of preferences as measured by the use of response categories and the same order of item difficulty. Further, a criterion of equivalence is met when similar correlation patterns between items occur in comparisons between cultures. However, even where we have followed a valid translation procedure there is no way of knowing a priori if the language chosen in the research protocol has the same meaning in different societies (Poortinga, 1989). If the protocol means different things it is not possible to make valid comparisons.

Underlying all measurements concerns are the concepts of reliability and validity. Reliability is assessed by several methods, but refers essentially to whether the instrument elicits consistent responses internally or over time. Internal reliability is determined by intercorrelations that measure the degree to which items belong together. Significant intercorrelations are expected where measures have structural equivalence. Validity on the other hand asks the question of whether the research instrument measure what it purports to measure.

An important consideration is equivalence in the theoretical framework used or in the constructs being employed in the study. Comparability is impossible when the theoretical framework is not the same in comparison studies for example when the constructs measured mean different things depending on interpretation. Typically respondents in Western Europe or North America are guided by rationality in responding to psychological assessments. Responses of Western participants are mainly influenced by an educational system that reinforces rationality and rewards logical thinking. Other cultures may reward intuitive thinking processes and that difference makes comparisons of cultures difficult. Different thinking processes between cultures may result in measuring an aspect of the culture that confounds the variable of interest and makes analysis guarded.

Van de Vijver and Leung (1997a, 1997b) suggested several types of linguistic equivalence. One approach asks whether the instrument measures the same underlying construct in the cultures examined. For example is the concept of intelligence the same in the comparative cultures, or does it depend on cultural uniqueness factors not measured? However, as noted above, a test of structural equivalence is found if the pattern of intercorrelations is the same or similar across cultures.

The underlying response structure in comparative studies can also be examined by means of factor analysis. A factor analysis with varimax rotation will allow the researcher to determine the level of factorial agreement. Factor analysis group survey items together based on item intercorrelations, and the factors are thought to represent different independent concepts. If the same group of items emerge in different cultural samples that is taken as evidence of structural similarity or structural equivalence. The similarity of the factor loadings is accepted as evidence of psychometric equivalence. Statistical techniques to further assist in evaluating psychometric equivalence include regression analysis that reports the amount of variance contributed by each variable to the construct. In the analysis of variance procedure it is also possible to examine for interaction between items and cultural effects, and where none are present that result is evidence of equivalence.

Date: 2015-01-11; view: 939