Psychology in Everyday Life: Critically Evaluating the Validity of Websites
The validity of research reports published in scientific journals is likely to be high because the hypotheses, methods, results, and conclusions of the research have been rigorously evaluated by other scientists, through peer review, before the research was published. For this reason, you will want to use peer-reviewed journal articles as your major source of information about psychological research.
Although research articles are the gold standard for validity, you may also need and desire to get at least some information from other sources. The Internet is a vast source of information from which you can learn about almost anything, including psychology. Search engines—such as Google or Yahoo!—bring hundreds or thousands of hits on a topic, and online encyclopedias, such as Wikipedia, provide articles about relevant topics.
Although you will naturally use the web to help you find information about fields such as psychology, you must also realize that it is important to carefully evaluate the validity of the information you get from the web. You must try to distinguish information that is based on empirical research from information that is based on opinion, and between valid and invalid data. The following material may be helpful to you in learning to make these distinctions.
The techniques for evaluating the validity of websites are similar to those that are applied to evaluating any other source of information. Ask first about the source of the information. Is the domain a “.com” (business), “.gov” (government), or “.org” (nonprofit) entity? This information can help you determine the author’s (or organization’s) purpose in publishing the website. Try to determine where the information is coming from. Is the data being summarized from objective sources, such as journal articles or academic or government agencies? Does it seem that the author is interpreting the information as objectively as possible, or is the data being interpreted to support a particular point of view? Consider what groups, individuals, and political or commercial interests stand to gain from the site. Is the website potentially part of an advocacy group whose web pages reflect the particular positions of the group? Material from any group’s site may be useful, but try to be aware of the group’s purposes and potential biases.
Also, ask whether or not the authors themselves appear to be a trustworthy source of information. Do they hold positions in an academic institution? Do they have peer-reviewed publications in scientific journals? Many useful web pages appear as part of organizational sites and reflect the work of that organization. You can be more certain of the validity of the information if it is sponsored by a professional organization, such as the American Psychological Association or the American Psychological Society.
Try to check on the accuracy of the material and discern whether the sources of information seem current. Is the information cited such that you can read it in its original form? Reputable websites will probably link to other reputable sources, such as journal articles and scholarly books. Try to check the accuracy of the information by reading at least some of these sources yourself.
It is fair to say that all authors, researchers, and organizations have at least some bias and that the information from any site can be invalid. But good material attempts to be fair by acknowledging other possible positions, interpretations, or conclusions. A critical examination of the nature of the websites you browse for information will help you determine if the information is valid and will give you more confidence in the information you take from it.
· Research is said to be valid when the conclusions drawn by the researcher are legitimate. Because all research has the potential to be invalid, no research ever “proves” a theory or research hypothesis.
· Construct validity, statistical conclusion validity, internal validity, and external validity are all types of validity that people who read and interpret research need to be aware of.
· Construct validity refers to the assurance that the measured variables adequately measure the conceptual variables
· Statistical conclusion validity refers to the assurance that inferences about statistical significance are appropriate.
· Internal validity refers to the assurance that the independent variable has caused the dependent variable. Internal validity is greater when confounding variables are reduced or eliminated.
· External validity is greater when effects can be replicated across different manipulations, measures, and populations. Scientists use meta-analyses to better understand the external validity of research.
EXERCISES AND CRITICAL THINKING
1. The Pepsi Cola Corporation, now PepsiCo Inc., conducted the “Pepsi Challenge” by randomly assigning individuals to taste either a Pepsi or a Coke. The researchers labeled the glasses with only an “M” (for Pepsi) or a “Q” (for Coke) and asked the participants to rate how much they liked the beverage. The research showed that subjects overwhelmingly preferred glass “M” over glass “Q,” and the researchers concluded that Pepsi was preferred to Coke. Can you tell what confounding variable is present in this research design? How would you redesign the research to eliminate the confound?
2. Locate a research report of a meta-analysis. Determine the criteria that were used to select the studies and report on the findings of the research.
 Stangor, C. (2011). Research methods for the behavioral sciences (4th ed.). Mountain View, CA: Cengage.
 Nunnally, J. C. (1978). Pyschometric theory. New York, NY: McGraw-Hill.
 Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
 Rosenthal, R., & Fode, K. L. (1963). The effect of experimenter bias on the performance of the albino rat. Behavioral Science, 8, 183–189.
 Heine, S. J. (2010). Cultural psychology. In S. T. Fiske, D. T. Gilbert, & G. Lindzey (Eds.),Handbook of social psychology (5th ed., Vol. 2, pp. 1423–1464). Hoboken, NJ: John Wiley & Sons.
2.4 Chapter Summary
Psychologists study the behavior of both humans and animals in order to understand and improve the quality of human lives.
Psychological research may be either basic or applied in orientation. Basic research and applied research inform each other, and advances in science occur more rapidly when both types of research are conducted.
The results of psychological research are reported primarily in research reports in scientific journals. These research reports have been evaluated, critiqued, and improved by other scientists through the process of peer review.
The methods used by scientists have developed over many years and provide a common framework through which information can be collected, organized, and shared.
The scientific method is the set of assumptions, rules, and procedures that scientists use to conduct research. In addition to requiring that science be empirical, the scientific method demands that the procedures used be objective, or free from personal bias.
Scientific findings are organized by theories, which are used to summarize and make new predictions, but theories are usually framed too broadly to be tested in a single experiment. Therefore, scientists normally use the research hypothesis as a basis for their research.
Scientists use operational definitions to turn the ideas of interest—conceptual variables—into measured variables.
Decisions about whether psychological research using human and animals is ethical are made using established ethical codes developed by scientific organizations and on the basis of judgments made by the local Institutional Review Board. These decisions are made through a cost-benefit analysis, in which the costs are compared to the benefits. If the potential costs of the research appear to outweigh any potential benefits that might come from it, then the research should not proceed.
Descriptive research is designed to provide a snapshot of the current state of affairs. Descriptive research allows the development of questions for further study but does not assess relationships among variables. The results of descriptive research projects are analyzed using descriptive statistics.
Correlational research assesses the relationships between and among two or more variables. It allows making predictions but cannot be used to draw inferences about the causal relationships between and among the variables. Linear relationships between variables are normally analyzed using the Pearson correlation coefficient.
The goal of experimental research is to assess the causal impact of one or more experimental manipulations on a dependent variable. Because experimental research creates initial equivalence among the participants in the different experimental conditions, it allows drawing conclusions about the causal relationships among variables. Experimental designs are not always possible because many important variables cannot be experimentally manipulated.
Because all research has the potential for invalidity, research never “proves” a theory or hypothesis.
Threats to construct validity involve potential inaccuracies in the measurement of the conceptual variables.
Threats to statistical conclusion validity involve potential inaccuracies in the statistical testing of the relationships among variables.
Threats to internal validity involve potential inaccuracies in assumptions about the causal role of the independent variable on the dependent variable.
Threats to external validity involve potential inaccuracy regarding the generality of observed findings.
Informed consumers of research are aware of the strengths of research but are also aware of its potential limitations.