8.2.1: Natural Groups Research

Learning Objectives

1. Explain why a researcher might choose to conduct natural groups design rather than an experimental design that compares randomly assigned IV groups.
2. Explain why correlation does not imply causation.

What if there's no random assignment?

Experimental designs with random assignment of participants into the IV groups are the best way to show that an IV group caused changes in the DV. This is because everything else that could have caused changed in the DV should have been distributed randomly across the different IV groups. There are some ideas to unpack here. First, the other potential causes should have be randomly distributed, but unless you measure these potential causes then you can't be sure; random assignment is based on chance. Second, this type of research methodology's purpose is to try to support the hypothesis that one variable caused changes in another variable, but sometimes it's less important to know what is the cause if we know that the variables are somehow related.

Sometimes we want to show that one variable caused changes in the other variable, but what we think is the cause can't be randomly assigned. When the researcher doesn't have control over the IV and can't randomly assign participants, then the research is a natural groups design; these types of designs are sometimes called posttest-only nonequivalent groups design, but that name will be discussed later in the chapter on quasi-experimental designs. This is most common when what the researcher thinks is the cause is either part of the participant (like their social group or their opinion) or it is something that the participant chose. For example, as a researcher, we cannot randomly assign participants into a demographic group (like race, gender, sexual orientation, etc.) or whether they like pineapples on pizza or not. When choice is involved, sometimes the variable could be randomly assigned, but it is not. For example, if an instructor wanted to know if eating before an exam improved performance, they could either randomly assign students into two groups (eat before the exam or not) and expect the students to follow the directions of their assigned group, or the instructor could ask students if they ate before the exam (no random assignment; the example with random assignment is an experiment, while the example in which the students were merely asked about their eating would be a natural groups design. The distinction is important because if the study was an experiment, then it could be concluded that making eating before can exam improves grades (cause-and-effect), but if it was a natural groups design, we could only conclude that these variables are related. Instead of eating being what caused better exam performance, it could be that students who have more time to eat before an exam also have more time to study before an exam; thus, while there could be a relationship between eating and exam performance, it might be that the real cause of exam performance is study time.

There are no IVs in natural groups designs (technically), but we still use the term IV.

An important note here about independent variables: the definition of an IV is "the variable that the experimenter manipulates, and believes is the cause of changes in the outcome variable," which means that there are technically no IVs in natural groups designs. However, researchers often use the term IV because they think that the one variable is the cause of changed in the other, even if their study does not randomly assign participants into the groups. Plus, it's just easier to use the terminology that we're used to! But by definition, in natural groups designs we have predictor variables and outcome variables, not IVs and DVs.

Natural Groups or Correlational Research?

Many social science researchers use the term correlational research to describe studies that do not have random assignment. Dr. MO chose to use the term natural groups design to avoid the common misconception that correlational research must include a correlational statistical analysis (which requires at least two quantitative variables). The defining feature of natural groups designs is that the predictor and outcome variables are measured, neither one is manipulated. This is true regardless of whether the variables are quantitative or categorical. Imagine, for example, that a researcher administers the Rosenberg Self-Esteem Scale to 50 American college students and 50 Japanese college students. Although this “feels” like an experiment, it is a natural groups design because the researcher did not (and cannot) manipulate the students’ nationalities. The defining feature of natural groups designs research is that no variables are manipulated. It does not matter how or where the variables are measured. A researcher could have participants come to a laboratory to complete a computerized backward digit span task and a computerized risky decision-making task and then assess the relationship between participants’ scores on the two tasks. Or a researcher could go to a shopping mall to ask people about their attitudes toward the environment and their shopping habits and then assess the relationship between these two variables. Both of these studies could be natural groups designs because no independent variable is manipulated. The crucial point is that what defines a study as experimental or natural groups is not the variables being studied nor the type of statistics used to analyze the data; what defines a study is how the study is conducted.

Correlation Does Not Imply Causation

Remember in the Analysis chapter when you were reminded of correlations, and there was a section talking about correlations versus causation? This idea becomes relevant again when discussing natural groups designs. An amusing example of this comes from Messerli's (2012) study that showed a positive correlation (Pearson’s r = 0.79) between the per capita chocolate consumption of a nation and the number of Nobel prizes awarded to citizens of that nation. It seems clear, however, that this does not mean that eating chocolate causes people to win Nobel prizes, or that Nobel prizes cause citizens to eat more chocolate! There are two reasons that showing that two variables are related does not mean that the predictor causes the outcome.

The first is called the directionality problem. Two variables, X and Y, can be related because X causes Y or because Y causes X. This means that one variable may be causing changes in the other variable, the research design cannot tell is which one is the causes. For example, we might find that exercise is related to how happiness such that people who exercise are happier on average than people who do not. This relationship is consistent with the idea that exercising causes happiness, but it is also consistent with the idea that happiness causes exercise. Perhaps being happy gives people more energy or leads them to seek opportunities to socialize with others by going to the gym. With a natural groups design, we cannot be sure which is the causal variable.

The second reason that showing that two variables are related does not mean that the predictor causes the outcome is called the third variable problem. Two variables, X and Y, can be statistically related not because X causes Y, or because Y causes X, but because some third variable, Z, causes both X and Y. Figure \(\PageIndex{1}\) is a poster created by one of Dr. MO's students to visual this third variable problem, which the class started to refer to as "What about Z?".

Figure \(\PageIndex{1}\): What about Z? by Laurel Woodruff CC-BY-NC-SA

For example, the fact that nations that have won more Nobel prizes tend to have higher chocolate consumption probably reflects geography in that European countries tend to have higher rates of per capita chocolate consumption and invest more in education and technology (once again, per capita) than many other countries in the world. Similarly, the relationship between exercise and happiness could mean that some third variable, such as social class, causes both of the others. Having disposabile income could allow people more time to exercise while also leading to less stress (and more happiness).

Some excellent and amusing examples of spurious correlations can be found at http://www.tylervigen.com (Figure \(\PageIndex{2}\) provides one such example).

Figure \(\PageIndex{2}\): Example of a Spurious Correlation. Source: http://tylervigen.com/spurious-correlations (CC-BY 4.0)

“Lots of Candy Could Lead to Violence”

Although researchers in psychology know that correlation does not imply causation, many journalists do not. One website about correlation and causation, http://jonathan.mueller.faculty.noctrl.edu/100/correlation_or_causation.htm, links to dozens of media reports about real biomedical and psychological research. Many of the headlines suggest that a causal relationship has been demonstrated when a careful reading of the articles shows that it has not because of the directionality and third-variable problems.

One such article is about a study showing that children who ate candy every day were more likely than other children to be arrested for a violent offense later in life. But could candy really “lead to” violence, as the headline suggests? What alternative explanations can you think of for this statistical relationship? How could the headline be rewritten so that it is not misleading?

As you have learned by reading this book, there are various ways that researchers address the directionality and third-variable problems. The most effective is to conduct an experiment. For example, instead of simply measuring how much people exercise, a researcher could bring people into a laboratory and randomly assign half of them to run on a treadmill for 15 minutes and the rest to sit on a couch for 15 minutes. Now, if the exercisers end up in more positive moods than those who did not exercise, it cannot be because their moods affected how much they exercised (because it was the researcher who used random assignment to determine how much they exercised). Likewise, it cannot be because some third variable (social class) affected both how much they exercised and what mood they were in. Experiments eliminate the directionality and third-variable problems and allow researchers to draw firm conclusions about causal relationships.

Why choose a natural groups design?

There are a few reasons that researchers might choose to conduct a natural groups design rather than an experiment. The first is that they are not interested in determining if there is a causal relationship between variables. Recall two goals of science are to describe and to predict; natural groups designs allow researchers to achieve both of these goals. Specifically, this strategy can be used to describe the statistical relationship between variables, and (if there is a relationship between the variables) the researchers can use scores on one variable to predict scores on the other.

Another reason that researchers would choose to use a natural groups rather than an experiment is that the they do believe that the predictor variable (sorta like an IV) does cause changed in the outcome variable (DV), but the researcher cannot manipulate the independent variable because it is impossible, impractical, or unethical. For example, while a researcher might be interested in the relationship between the frequency people use cannabis and their memory abilities they cannot ethically manipulate the frequency that people use cannabis. As such, they must rely on natural groups by asking about the frequency that people use cannabis and measure their memory abilities using a standardized test of memory to determine if there is a predictive relationship between the frequency people use cannabis and memory test performance.

Another strength of natural groups designs is that it is often higher in external validity than experimental research. Recall there is typically a trade-off between internal validity and external validity. As greater controls are added to experiments, internal validity is increased but often at the expense of external validity as artificial conditions are introduced that do not exist in reality. Natural group design may have low internal validity because nothing is manipulated or controlled but high external validity. Since nothing is manipulated or controlled by the experimenter, the results are more likely to reflect relationships that exist in the real world.

Finally, extending upon this trade-off between internal and external validity, correlational research can help to provide converging evidence for a theory. If a theory is supported by a true experiment that is high in internal validity as well as by a natural groups design that is high in external validity, then the researchers can have more confidence in the validity of their theory. As a concrete example, correlational studies establishing that there is a relationship between watching violent television and aggressive behavior have been complemented by experimental studies confirming that the relationship is a causal one (Bushman & Huesmann, 2001).

---

References

Bushman, B. J., & Huesmann, L. R. (2001). Effects of televised violence on aggression. In D. Singer & J. Singer (Eds.), Handbook of children and the media (pp. 223–254). Thousand Oaks, CA: Sage.

Messerli, F. H. (2012). Chocolate consumption, cognitive function, and Nobel laureates. New England Journal of Medicine, 367, 1562-1564.