Now we’re going to perform a simple statistical analysis of the N400 amplitude scores that you obtained in the previous exercise. We have two amplitude scores for each participant, one for related targets and one for unrelated targets, and we want to know if the scores are significantly different for these two experimental conditions. The simplest way to do this is with a paired t test.
I used the free JASP statistical package to run the t test, but you can use whatever package you find comfortable. Make sure you specify a paired t test rather than an independent-samples t test. The results are shown in Screenshot 3.8. Before you look at the t and p values, you should always look at the descriptive statistics. Once we get to more complex analyses, it will be really easy to make mistakes in the statistical analysis. The most common mistake is to incorrectly specify which variable is in which column of the data file. For example, you might think that the unrelated and related targets are stored in the first and second columns, respectively, reversing the actual order. This kind of error becomes both more likely and more likely to lead to incorrect conclusions when your design has several factors and each row of the data file has a dozen or more columns. By comparing the group means from the statistical analysis to the grand average waveforms, you can often detect these errors.
Screenshot 3.8
If you look at the group means in Screenshot 3.8, you’ll see a mean of 9.657 µV for the related targets (Bin 3) and 1.583 µV for the unrelated targets (Bin 4). Those values at least approximately match what you can see for the CPz channel from 300-500 ms in the grand average waveforms shown in Screenshot 3.4.
Now that we’ve verified that the descriptive statistics look correct, we can look at the t and p values. The effect was significant at the p < .001 level, and the effect size (Cohen’s d) was huge. The effect size of 3.145 indicates that the difference between the group means for related and unrelated targets was 3.145 times as large as the standard deviation of the scores. You won’t find effects this large in most experiments, but the N400 ERP CORE experiment was carefully designed to maximize the experimental effects, and we chose a paradigm that was known to produce large effects. Also, I chose 10 participants with really clear effects for the exercises in this chapter; the effect size was “only” 2.33 in the full sample of 40 participants (but this was still a huge effect size).
Limits on Comparing Descriptive Statistics with Grand Average Waveforms
When we score the amplitude of an ERP component as the mean voltage in a fixed time window, we can directly compare the group mean values from the statistical analysis with the grand average ERP waveforms. This is because this scoring method is a linear operation (for a definition and more information, see the Appendix in Luck, 2014). The order of operations does not matter for linear operations. This means that we can obtain our mean amplitude score from the single-subject waveforms and then compute the mean of these scores, and we will get exactly the same value that we would obtain by scoring the mean amplitude from the grand average waveform.
Unfortunately, most other scoring methods are not linear. For example, the peak amplitude in a given time window is not linear. If we obtain the peak amplitude from the single-subject waveforms and then compute the mean of these scores, the result will not be the same as the peak amplitude of the grand average waveform. However, you should still compare the group means from your statistical analysis with the grand average waveforms. If there is a large mismatch, then you may have made an error in specifying the order of variables in your statistical analysis, or your grand average waveform may not adequately represent what is happening at the single-subject level. In either case, you want to know!
