Although
there are numerous terms and concepts used to describe validity, we can
simplify the complexity of validity by organizing terms and concepts into four
main types: construct validity, external validity, internal validity, and statistical
conclusion validity. Mastering these Big 4 types of validity provides a
comprehensive framework to judge the strengths and weaknesses of scientific
research.
Determining
the overall validity of research is simply not a black-and-white decision.
Instead, we have four types of validity to reinforce the notion that we have
many different ways to determine the strength of a research study. In fact, rarely will you ever encounter a
study that ranks strong in all four validities. Remember, the methodological decisions
made to increase one type of validity often harm another type of validity
(e.g., steps taken to increase internal validity often harm external validity),
and similar tradeoffs can occur among all four of our major classes of validity
(Mitchell & Jolley, 2006 ).
In
addition, many studies are conducted that do not address all four validities.
Recall that only an experimental research methodology is able to establish
internal validity and cause-and-effect conclusions. Thus, when non-experimental
research methodologies are adopted, a study by default will be weak in internal
validity. Similarly, many researchers have aptly noted that external validity
can be a premature goal of research (Mook, 1983). Researchers may first want to
study a phenomenon in isolated populations or isolated situations before trying
to generalize their results to the population from which they drew their
sample. Therefore, to determine the
overall merit of a study, we initially need to recognize what the goals of the
study being conducted are and what validities need to be established to meet
those goals. The next step would be to weigh the strengths that make the study
valid against the weaknesses that threaten the study’s validity.
Because
individual studies are likely to be weak on one or more types of validity,
researchers rarely conduct and report single studies. Instead, they often
conduct multiple studies to answer the same research question using different
research methodologies. A validity weakness threatening the conclusions of one
study can be tested in a new study using a different methodology to address (or
remove) that potential threat to validity. A common practice is for researchers
to engage in systematic research in which a series of studies are conducted and
published over time. Alternatively, researchers will report multiple studies in
a single professional article to demonstrate how different threats to validity
were tested and ruled out across different studies of the article.
A common area of
confusion that can occur when learning about validity is determining which type
of validity is being threatened. We have found that students particularly have
difficulty when trying to distinguish construct validity problems from internal
validity problems. Our first tip is to highlight the fundamental difference
between these two ways to critique a research study. A critique of construct validity
involves thinking about each variable of a study separately, and then making a
judgment on whether each variable was measured or manipulated validly. If you have
concerns that a particular measure or manipulation was biased in representing
what it was supposed to represent, then your concern involves a critique about
construct validity. In contrast, internal validity involves thinking about the
relationships among variables of the study, and then making a judgment that one
or more of them can be clearly linked to causing other variables in the study.
(From "21st Century Psychology: A Reference Handbook" by Buskist and Davis (2008), Sage Publications)
