6.5.1: Effect Sizes

Learning Objectives

1. Interpret effects sizes.

Now that you've remembered a little bit about how to statistically test for differences between means, let's look at one number (an effect size) that can help you quantify these mean differences across studies.

Mean Differences Between Groups

Differences between groups or conditions are usually described in terms of the mean of each group or condition. For example, Ollendick et al. (2009) conducted a study in which they evaluated two one-session treatments for simple phobias in children. Ollendick et al. (2009) randomly assigned children with an intense fear (e.g., to dogs) to one of three conditions. In the exposure condition, the children actually confronted the object of their fear under the guidance of a trained therapist. In the education condition, they learned about phobias and some strategies for coping with them. In the wait-list control condition, they were waiting to receive a treatment after the study was over. There were several dependent variables, but we will look at the severity of each child’s phobia. This DV was operationally defined as a rating on a 1-to-8 scale by a clinician who did not know which treatment the child had received. The average fear rating in the education condition was 4.83 with a standard deviation of 1.52, while the mean fear rating in the exposure condition was 3.47 with a standard deviation of 1.77. The mean fear rating in the control condition was 5.56 with a standard deviation of 1.21. Is seems like both treatments worked, but the exposure treatment worked better than the education treatment. Differences between group or condition means can be presented in a bar graph like that in Figure \(\PageIndex{1}\), where the heights of the bars represent the group or condition means.

Figure \(\PageIndex{1}\): Bar Graph Showing Mean Clinician Phobia Ratings for Children in Two Treatment Conditions

Effect Sizes

It is also important to be able to describe the strength of a statistical relationship, which is often referred to as the effect size. The most widely used measure of effect size for differences between group or condition means is called Cohen’s d, which conceptually is the difference between the two means divided by a combined standard deviation (called the pooled-within groups standard deviation). Conceptually (but not quite mathematically), Cohen’s d is the difference between the two means expressed in standard deviation units. A Cohen’s d of 0.50 means that the two group means differ by 0.50 standard deviations (half a standard deviation). A Cohen’s d of 1.20 means that they differ by 1.20 standard deviations.

But how should we interpret these values in terms of the strength of the relationship or the size of the difference between the means? Table \(\PageIndex{1}\) presents some guidelines for interpreting Cohen’s d values in psychological research (Cohen, 1992). Values near 0.20 are considered small, values near 0.50 are considered medium, and values near 0.80 are considered large. Thus a Cohen’s d value of 0.50 represents a medium-sized difference between two means, and a Cohen’s d value of 1.20 represents a very large difference in the context of psychological research. In Ollendick et al. (2009), there was a large difference (d = 0.82) between the exposure and education conditions.

Table \(\PageIndex{1}\): Guidelines for Referring to Cohen’s d and Pearson’s r Values as “Strong,” “Medium,” or “Weak”

Relationship strength	Cohen’s ^d	Pearson’s ^r
Strong/large	0.80	± 0.50
Medium	0.50	± 0.30
Weak/small	0.20	± 0.10

Cohen’s d is useful because it has the same meaning regardless of the variable being compared or the scale it was measured on. A Cohen’s d of 0.20 means that the two group means differ by 0.20 standard deviations whether we are talking about scores on the Rosenberg Self-Esteem scale, reaction time measured in milliseconds, number of siblings, or diastolic blood pressure measured in millimeters of mercury. Not only does this make it easier for researchers to communicate with each other about their results, it also makes it possible to combine and compare results across different studies using different measures.

Be aware that the term effect size can be misleading because it suggests a causal relationship—that the difference between the two means is an “effect” of being in one group or condition as opposed to another. Imagine, for example, a study showing that a group of exercisers is happier on average than a group of non-exercisers, with an “effect size” of d = 0.35. If the study was an experiment—with participants randomly assigned to exercise and no-exercise conditions—then one could conclude that exercising caused a small to medium-sized increase in happiness. If the study was cross-sectional, however, then one could conclude only that the exercisers were happier than the nonexercisers by a small to medium-sized amount. In other words, simply calling the difference an “effect size” does not make the relationship a causal one.

Sex Differences Expressed as Cohen’s d

Hyde (2007) looked at the results of numerous studies on gender differences and expressed the results in terms of Cohen’s d . Following are a few of the values she has found, averaging across several studies in each case. (Note that because she always puts the mean for men first and subtracts the mean for women so positive values indicate that men score higher and negative values indicate that women score higher.)

Mathematical problem solving	+0.08
Reading comprehension	−0.09
Smiling	−0.40
Aggression	+0.50
Attitudes toward casual sex	+0.81
Leadership effectiveness	−0.02

Hyde (2007) points out that although men and women differ by a large amount on some variables (e.g., attitudes toward casual sex), they differ by only a small amount on the vast majority. In many cases, Cohen’s d is less than 0.10, which she terms a “trivial” difference. (The difference in talkativeness discussed in Chapter 1 was also trivial: d = 0.06.) Although researchers and non-researchers alike often emphasize gender differences, Hyde has argued that it makes at least as much sense to think of men and women as fundamentally similar. She refers to this as the “gender similarities hypothesis.”

We'll now move from comparing mean differences to testing for (linear) relationships with correlations.

---

References

Carlson, K. A., & Conard, J. M. (2011). The last name effect: How last name influences acquisition timing. Journal of Consumer Research, 38(2), 300-307. doi: 10.1086/658470

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.

Hyde, J. S. (2007). New directions in the study of gender similarities and differences. Current Directions in Psychological Science, 16, 259–263.

Ollendick, T. H., Öst, L.-G., Reuterskiöld, L., Costa, N., Cederlund, R., Sirbu, C.,…Jarrett, M. A. (2009). One-session treatments of specific phobias in youth: A randomized clinical trial in the United States and Sweden. Journal of Consulting and Clinical Psychology, 77, 504–516.