9.2: Quasi-Experimental Designs with One Group

Learning Objectives

1. Describe at least one way that quasi-experimental designs different from experimental designs.
2. Describe how interrupted time series designs are similar and different from pretest-posttest designs.

Recall that with a true experiment, random assignment to conditions is used to ensure the groups are equivalent or counterbalancing is used to guard against order effects. Quasi-experiments are missing one of these safeguards. In quasi-experimental designs, an independent variable is manipulated, but either a control group is missing or participants are not randomly assigned to conditions (Cook & Campbell, 1979). In this section, we will discuss quasi-experimental designs that compare one group on scores before and after something happens. The following section will discuss similar quasi-experimental designs but that include two or more groups.

One-Group Pretest-Posttest Design

In a one-group pretest-posttest design, the dependent variable is measured once before the treatment is implemented (called a pretest), and once after it is implemented (called a posttest). As described in the prelude, Hatzenbuehler et al. (2012) could be considered a one-group pretest-posttest design. Why? Because there was one group (sexual minority men) in which rates of the DV were measured before implementation of a treatment or event (the law was passed) and again after the event. This type of pretest-posttest design sounds a lot like a repeated measures design experiment because the participants are measured more than once; each participant is tested first under the control condition and then under the treatment condition. However, it is not a repeated measures design because the order of conditions is not counterbalanced. In Hatzenbuehler et al. (2012), the event was legal recognition of same-gender marriage, but we cannot randomly assign the order of time (before/after).

The implementation of the treatment or event is often referred to as an IV because we think that it caused changes in the DV, but technically these are not IVs because the definition of independent variables is "the variable that the experimenter manipulates." In quasi-experimental designs, the experimenter was not in control of the IV. So even if if the average posttest score is statistically different than the average pretest score, we cannot conclude that the treatment is responsible for the change because there may be other explanations for why the posttest scores may have changed. These alternative explanations pose threats to internal validity, and are discussed later in this chapter.

Interrupted Time Series Design

A variant of the pretest-posttest design is the interrupted time-series design. A time series is a set of measurements taken at intervals over a period of time. This is like the one-group pretest-posttest design with the addition of measuring the outcome multiple times. For example, a manufacturing company might measure its workers’ productivity each week for a year. In an interrupted time series-design, a time series like this one is “interrupted” by a treatment. In one classic example, the treatment was the reduction of the work shifts in a factory from 10 hours to 8 hours (Cook & Campbell, 1979). Because productivity increased rather quickly after the shortening of the work shifts, and because it remained elevated for many months afterward, the researcher concluded that the shortening of the shifts caused the increase in productivity. Notice that the interrupted time-series design is like a pretest-posttest design in that it includes measurements of the dependent variable both before and after the treatment. It is unlike the pretest-posttest design, however, in that it includes multiple pretest and posttest measurements.

Figure \(\PageIndex{1}\) shows data from a hypothetical interrupted time-series study. The dependent variable is the number of student absences per week in a research methods course. The treatment is that the instructor begins publicly taking attendance each day so that students know that the instructor is aware of who is present and who is absent. The top panel of Figure \(\PageIndex{1}\) shows how the data might look if this treatment worked. There is a consistently high number of absences before the treatment, and there is an immediate and sustained drop in absences after the treatment. The bottom panel of Figure \(\PageIndex{1}\) shows how the data might look if this treatment did not work. On average, the number of absences after the treatment is about the same as the number before. This figure also illustrates an advantage of the interrupted time-series design over a simpler pretest-posttest design. If there had been only one measurement of absences before the treatment at Week 7 and one afterward at Week 8, then it would have looked as though the treatment were responsible for the reduction. The multiple measurements both before and after the treatment suggest that the reduction between Weeks 7 and 8 is nothing more than normal week-to-week variation.

Figure \(\PageIndex{1}\): A Hypothetical Interrupted Time-Series Design. The top panel shows data that suggest that the treatment caused a reduction in absences. The bottom panel shows data that suggest that it did not.

The prelude didn't get into details, but because Hatzenbuehler et al. (2012) measured medical care visits, medical care costs, mental health care visits, and mental health care costs over a two year period, we can assume that the results could have been analyzed as an interrupted time series design. Instead, Hatzenbuehler et al. (2012) averaged the costs and visits for each time period for their comparisons.

---

References

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings. Boston, MA: Houghton Mifflin.

Hatzenbuehler, M. L, O'Cleirigh, C., Grasso, C., Mayer, K., Safren, S., & Bradford, J. (2012). Effect of same-sex marriage laws on health care use and expenditures in sexual minority men: A quasi-natural experiment. American Journal of Public Health, 102(2), 285-291.