8.4: Repeated Measures Design

Learning Objectives

1. Explain the difference between between groups designs and repeated measures design, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
2. Explain how counterbalancing helps to deal with carryover effects.

In this section, we look at another way to design an experiment. The primary distinction from the between groups and matched groups design that we previously discussed is whether each participant experiences one level of the independent variable or each participant experiences all levels of the independent variable. The latter are called within subjects experiments, but the easiest to understand is the repeated measures design.

Repeated Measures Design

In repeated measures designs, each participant is tested under all conditions, but the order that each participant experiences each IV condition is randomized. Remember our between groups design comparing digital stress of participants who were randomly assigned into two groups: half of the participants were randomly assigned into the IV condition of limiting their social media use, while the other half were randomly assigned to not limit their social media use. In a repeated measures design, all participants would experience each of the IV conditions but the researcher would instead randomly assign the order of the IV conditions. Half of the participants would first be limited in their social media use for two weeks, and then those same participants could use as much social media as they'd like for two weeks. The other half of the participants would first use social media as much as they'd like for two weeks, and then be limited in their social media use for two weeks. At the end of each two week period, all participants would have their digital stress assessed.

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in each condition have the same anxiety levels, same socioeconomic status, same number of siblings, and so on, because they are the very same people. Repeated measures experiments, and matched groups designs, also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect.

Your Research Ideas: Between Groups, Matched, or Repeated Measures?

Not all experiments can use a repeated measures design, nor would it be desirable to do so. Think about an IV that a professional in your future career might be interested in investigating its effects on an outcome to help their clients. Based on that IV, what kind of design might work best?

• Between Groups design?
• Matched Groups design? What would be the matching variable? Note that the matching variable should not be the IV nor the DV, but might be expected to influence the DV.
• Repeated Measures design? Are you sure that the IV can be experienced in any order?

Counterbalancing

Counterbalancing is testing different participants in different orders. This was included in the definition of repeated measures designs, that the order of the IV conditions is randomized for each participants. The digital stress example above also included counterbalancing; half of the participants were asked to limit their social media use at the beginning of the study, and the other half of the participants were asked to limit their social media use at the end of the study.

Based on the need for counterbalancing, you can see that the primary disadvantage of repeated measures designs is that they can result in order effects. An order effect occurs when participants’ responses in the various conditions are affected by the order of conditions to which they were exposed. One type of order effect is a carryover effect. A carryover effect is an effect of being tested in one condition on participants’ behavior in later conditions. One type of carryover effect is a practice effect, where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect, where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This type of effect is called a context effect (or contrast effect). For example, digital stress may be higher for participants who first limited their social media use and then had unlimited access. In contrast to the social media break, the unlimited use might feel overwhelming. Note that this would not be the case for participants whose first IV condition was the unlimited condition. In other words, the order of the conditions may be a confounding variable. Thus any difference between the conditions in terms of the dependent variable could be caused by the order of the conditions and not the independent variable itself.

The best method of counterbalancing is complete counterbalancing in which an equal number of participants complete each possible order of conditions. However, for practical issues or due to the competency of the researcher, sometimes that does not happen. We've already covered an example when there are two IV conditions when discussing the the IV conditions of limited or unlimited social media use. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With four conditions, there would be 24 different orders; with five conditions there would be 120 possible orders. With counterbalancing, participants are assigned to orders randomly. Thus, random assignment plays an important role in repeated measures designs just as in between groups designs. Here, instead of randomly assigning to conditions, participants are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

A more efficient way of counterbalancing is partial counterbalancing through a Latin square design which randomizes through having equal rows and columns. For example, if you have four treatments, you must have four versions. Like a Sudoku puzzle, no treatment can repeat in a row or column. For four versions of four treatments, the Latin square design would look like Table \(\PageIndex{1}\):

Table \(\PageIndex{1}\): Example of Latin Square for counterbalancing

A	B	C	D
B	C	D	A
C	D	A	B
D	A	B	C

In Table \(\PageIndex{1}\), each condition appears at each ordinal position (A appears first once, second once, third once, and fourth once) and each condition precedes and follows each other condition one time. A Latin square for an experiment with 6 conditions would by 6 x 6 in dimension, one for an experiment with 8 conditions would be 8 x 8 in dimension, and so on. So while complete counterbalancing of 6 conditions would require 720 orders, a Latin square would only require 6 orders.

When the number of conditions is large, experiments can use random counterbalancing in which the order of the conditions is randomly determined for each participant. Using this technique, every possible order of conditions is determined and then one of these orders is randomly selected for each participant. This is not as powerful a technique as complete counterbalancing or partial counterbalancing using a Latin squares design. Use of random counterbalancing will result in more random error, but if order effects are likely to be small and the number of conditions is large, this is an option available to researchers.

There are two ways to think about what counterbalancing accomplishes. One is that it controls the order of conditions so that it is no longer a confounding variable. Instead of the limited social media condition always being first and the unlimited condition always being second, the limited social media condition comes first for some participants and second for others. Likewise, the unlimited social media condition comes first for some participants and second for others. Thus, any overall difference in the dependent variable of digital stress between the two conditions cannot have been caused by the order of conditions. A​​​​​ second way that counterbalancing is helpful is that if there are carryover effects, counterbalancing makes it possible to detect them because the different orders can be included in statistical analyses. One can analyze the data separately for each order to see whether it had an effect.

Disadvantages of Repeated Measures Designs

In addition to order effects, another disadvantage of repeated measures designs could be that it might be easier for participants to guess the hypothesis. For example, a participant who is asked about their digital stress after two limiting and two weeks not limiting their social media use (in whatever order) may recognize that the research is about social media's effect on digital stress. This knowledge could lead participants to rate their digital stress in the direction that they think is expected.

Type of Design

Many research topics can be studied using a between groups design, matched group design or a repeated measures design, so you must decide which of the approaches best fits the research question based on the design's relative merits for the particular situation.

Between groups designs have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Matched group designs also avoid carryover effects, but take a little more time and effort because the matching variable must be measured and then the participants must be matched then randomly assigned into an IV condition. Repeated measures designs have the advantage of controlling more extraneous participant variables than either matched groups or between groups, which generally reduces noise in the data and makes it easier to detect any effect of the independent variable upon the dependent variable. Repeated measures designs also require fewer participants than the other designs to detect an effect of the same size.

There's no real right answer as the best design will differ based on your participants, IV, and DV. Remember also you are choosing one type of design for one experiment. This does not preclude using another type in a different study. Replication is necessary in research, and using different types of designs that tend to find the same results improves the internal validity of your findings.