The problems of validity.

The dogmatic preference for qualitative or quantitative methods is based on controversial arguments over the validity of results and subsequent interpretation. Validity is no small matter since the objective of all research is to demonstrate a real as opposed to a fantasized interpretation of cultural behavior. The issues of validity are best answered through the utilization of methods that combine all available approaches, that demonstrate conceptual clarity, and that ensures replication of the studies over time. The results of either cultural or cross-cultural research are always approximations toward validity as no method can yield a complete picture valid for all time (Cook & Campbell, 1979).

Qualitative research poses unique problems in establishing validity. Experiments are not easily created or conducted for situations where the investigator is trying to understand underlying customs and rules of the culture. Likewise standardized surveys or scales are difficult to construct at this basic level of investigations. Further the emic approach generally is focused on individual development in interaction with cultural norms and rules. These gradual psychological processes are not easily investigated using quantitative methods that assume the measurement of stable or static qualities. To capture the developmental changes some have argued for the use of video recordings (Greenfield, 1997a).

One form of qualitative validity is called interpretive and aims at establishing communication clarity between the researcher and the cultural group observed (Maxwell, 1992). Ecological validity is the extent to which data is collected consistent with the cultural context. Greenfield believes that is accomplished when studying behavior in the natural context. However, the major problem with validity in the qualitative approach is the inability of other researchers to verify (replicate) the results since selection of what to study as well as interpretations are the results of the subjective qualities of the researcher.

Experiments in cultural and cross-cultural research cannot control the experimental conditions, especially in the administration of “treatments” to a experimental group that can be properly compared to the behavior of a control group (Van de Vijver & Leung, 1997a, 1997b). However, there are ways to establish relative validity for differences between cultural groups. If cultural samples are selected for established ethnographic conditions rather than for convenience the differences between groups become more plausible. However, since any interpretation come after the completion of the experiment the possibility for incorrect interpretations remain a significant vulnerability. Still when differences are compared based on established theory where the group-related variables are treated as independent variables the possibility for erroneous interpretation is reduced (Malpass, 1977). When the sample is chosen on the basis of the presence of these group-related variables validity is implicit in the methodology. Various statistical techniques can also be employed to remove error as discussed earlier for example by means of regression analysis that allows for a comparison of the contribution of various concepts to the overall behavior in question.

To address validity the inclusion of separate and independent measures of the relationships being examined are also important. That would also include conducting separate testing at different points in time, but also replication work. The latter would establish the reliability of results that is a necessary precondition for validity. Along the same line different types of measurements should be encouraged. For example the use of surveys in conjunction with experimental work can give us some evidence of convergent validity. Likewise if the results from comparative studies are what would be predicted from other research and underlining theory, that too would indicate validity. In other words validity is confirmed when the results are as expected from the theoretical perspective. Finally, meta-analysis is also another approach to convergent validity.

Date: 2015-01-11; view: 859