8.5: Quasi-Experimental Research (Summary)

Key Takeaways

• Quasi-experimental research involves the manipulation of an independent variable without the random assignment of participants to conditions or counterbalancing of orders of conditions.
• There are three types of quasi-experimental designs that are within-subjects in nature. These are the one-group posttest only design, the one-group pretest-posttest design, and the interrupted time-series design.
• There are five types of quasi-experimental designs that are between-subjects in nature. These are the posttest only design with nonequivalent groups, the pretest-posttest design with nonequivalent groups, the interrupted time-series design with nonequivalent groups, the pretest-posttest design with switching replication, and the switching replication with treatment removal design.
• Quasi-experimental research eliminates the directionality problem because it involves the manipulation of the independent variable. However, it does not eliminate the problem of confounding variables, because it does not involve random assignment to conditions or counterbalancing. For these reasons, quasi-experimental research is generally higher in internal validity than non-experimental studies but lower than true experiments.
• Program evaluation is a form of applied research designed to assess the effectiveness of interventions. The quasi-experimental designs discussed in this chapter—particularly the pretest-posttest design with nonequivalent groups and interrupted time-series designs—are commonly employed in program evaluation research.
• Of all of the quasi-experimental designs, those that include a switching replication are highest in internal validity.

Key Terms and Concepts

QUASI-EXPERIMENTAL RESEARCH

Research that includes manipulation of an independent variable but lacks random assignment to conditions.

ONE-GROUP POSTTEST ONLY DESIGN

Dependent variable is measured once after treatment is implemented.

ONE-GROUP PRETEST-POSTTEST DESIGN

Dependent variable is measured once before treatment and once after.

HISTORY

External events occurring during a study that affect the dependent variable.

MATURATION

Natural changes in participants over time unrelated to treatment.

TESTING

Effects of taking a test on subsequent test performance.

INSTRUMENTATION

Changes in measurement procedures or instruments during a study.

REGRESSION TO THE MEAN

The tendency for extreme scores to become less extreme upon retesting.

SPONTANEOUS REMISSION

Natural improvement that occurs without treatment.

INTERRUPTED TIME-SERIES

When a set of measurements is taken over time, and interrupted by a treatment.

NONEQUIVALENT GROUPS DESIGN

Comparing groups that were not formed by random assignment.

POSTTEST ONLY NONEQUIVALENT GROUPS DESIGN

Comparing non-equivalent groups only after treatment.

INTERRUPTED TIME-SERIES DESIGN WITH NONEQUIVALENT GROUPS

Multiple measurements in both treatment and control groups.

PRETEST- POSTTEST DESIGN WITH SWITCHING REPLICATION DESIGN

A second group receives treatment after serving as control.

SWITCHING REPLICATION WITH TREATMENT REMOVAL DESIGN

Treatment is alternately applied and removed across conditions.

Test Your Knowledge (answers at end of section)

1. What is the primary limitation of the one-group pretest-posttest design?

A) It is too expensive to conduct

B) It lacks a control group, making it impossible to rule out alternative explanations like maturation, history, and testing effects

C) It requires random assignment

D) It cannot measure change over time

2. A researcher selects students who scored extremely high on a test of favorable attitudes toward illegal drugs and enrolls them in an anti-drug program. At the posttest, the students' attitudes are less favorable toward drugs. The researcher concludes the program was effective. What threat to internal validity is most likely responsible for this observed change, even if the program had no effect?

A) History

B) Instrumentation

C) Regression to the mean

D) Maturation

3. What distinguishes a non-equivalent groups design from a true experiment?

A) Non-equivalent groups designs use larger samples

B) Non-equivalent groups designs compare groups but lack random assignment, so groups may differ systematically before treatment

C) Non-equivalent groups designs are always conducted in laboratories

D) Non-equivalent groups designs never use control groups

4. A researcher implements an exercise intervention for one group of patients with depression while measuring depression levels in a comparison group of students with depression who don't receive the intervention. After one week, the researcher removes the exercise treatment from the patients and introduces it to the students. One week later, depression has decreased in students but increased in patients. What design is this, and why does it provide strong evidence for treatment effectiveness?

A) Pretest-posttest design with nonequivalent groups; it controls for history effects

B) Switching replication with treatment removal design; it demonstrates the treatment effect in two groups at different times AND shows reversal when treatment is removed

C) Interrupted time-series design; it uses multiple measurements over time

D) Posttest only design with nonequivalent groups; it avoids testing effects

Answer Key

1. B - It lacks a control group, making it impossible to rule out alternative explanations like maturation, history, and testing effects

The one-group pretest-posttest design measures the same group before and after a treatment, but without a control/comparison group, it cannot rule out alternative explanations for observed changes. Maturation (natural changes over time), history (external events), testing effects (practice from pretest), and other confounds could explain differences rather than the treatment. This lack of control makes causal conclusions impossible.

2. C - Regression to the mean

Regression to the mean refers to the statistical phenomenon where individuals who score extremely high or low on a variable on one occasion will tend to score less extremely on the next occasion, simply by chance. Because the researcher selected only students with extreme scores (extremely favorable attitudes toward drugs), their scores would naturally tend to move toward the average (less extreme) at posttest, even without any intervention. This is a classic example of how regression to the mean can be mistaken for a treatment effect when participants are selected based on their extreme scores.

3. B - Non-equivalent groups designs compare groups but lack random assignment, so groups may differ systematically before treatment

Non-equivalent groups designs compare two or more groups (like a treatment and control group) but participants are not randomly assigned to groups. Instead, pre-existing or naturally formed groups are used. Because of the lack of random assignment, groups may differ systematically before treatment begins, making it unclear whether posttest differences are due to the treatment or pre-existing differences. This is why these designs are quasi-experimental rather than true experiments.

4. B - Switching replication with treatment removal design; it demonstrates the treatment effect in two groups at different times AND shows reversal when treatment is removed

This is a switching replication with treatment removal design. It provides particularly strong evidence for treatment effectiveness for three reasons: (1) It demonstrates the treatment effect is replicable by showing it works in two different groups (patients first, then students), (2) It shows the effects are staggered over time (groups improve at different times corresponding to when they receive treatment), and (3) It demonstrates reversibility—when treatment is removed from the first group, their symptoms worsen, providing strong evidence that the treatment was responsible for the initial improvement.

References

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings. Boston, MA: Houghton Mifflin.

Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16, 319–324.

Posternak, M. A., & Miller, I. (2001). Untreated short-term course of major depression: A meta-analysis of studies using outcomes from studies using wait-list control groups. Journal of Affective Disorders, 66, 139–146.

Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy. Baltimore, MD: Johns Hopkins University Press.

Exercises

• Practice: Imagine that two professors decide to test the effect of giving daily quizzes on student performance in a statistics course. They decide that Professor A will give quizzes but Professor B will not. They will then compare the performance of students in their two sections on a common final exam. List five other variables that might differ between the two sections that could affect the results.
• Discussion: Imagine that a group of obese children is recruited for a study in which their weight is measured, then they participate for 3 months in a program that encourages them to be more active, and finally their weight is measured again. Explain how each of the following might affect the results:
  • regression to the mean
  • spontaneous remission
  • history
  • maturation