Skip to main content
Social Sci LibreTexts

5.5: Use of Time Series Designs for Casual Inference

  • Page ID
    10353
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    What do you mean more? Weren’t we excited to be adding just one more time of measurement?

    Well, two is qualitatively different from one, so that’s good. But more really is merrier. Additional times of measurement allow researchers to look at these predictors of change across additional time gaps (from a predictor at Time 2 to changes form Time 2 to Time 3), to use growth curves to look at trajectories of mean level change (from a predictor at Time 1 to a trajectory from Time 1 to Time n),

    Can we even use these designs to look at how changes in our antecedents predict changes in our outcomes? Yes, these have been called “Change-to-change” models (Skinner et al., 1998), but you should stop using the word “predict” to describe these connections. They are actually like correlations between growth curves, and then we land back in our concurrent correlations puddle. So it would probably be better to look at the connections between time-lagged growth curves, for example, the connection between a growth curve from Time 1 to Time n to predict an outcome from Time 2 to Time n+1 (see Figure 23.5). Then you could also look at reciprocal change-to-change effects by switching around your antecedents and consequences. Just like with the designs that incorporate only two measurement points, these two analyses can provide different estimates of the connections between the different portions of the growth curves.

    ----------------------------------- Insert Figure 23.5 about here -----------------------------------

    What is happening with all our third variables in these analyses?

    We can control for them, but it may be more interesting to look directly at their effects, by organizing our data into a “niche” study, where we look at the connections we are interested in for subgroups of children—for boys or girls separately or for students high or low in achievement. If the connection between teacher involvement and student engagement is due to gender, it will disappear when we look at boys and girls separately. Looking directly at the developmental patterns of different groups is more consistent with a relational meta-theory which holds that development is differentially shaped by the characteristics of the people and their contexts (see box).

    Perceived control, teacher support, engagement, and achievement.

    An example of a “niche” design using growth curves can be seen in a study that examined the connections between teacher context (involvement and structure), students’ perceived control, their engagement, and achievement from the beginning of grade 3 to the end of grade 7 (Skinner et al., 1998). As you would expect, these variables were positively intercorrelated with each other at each time point and also predicted changes in each other over time.

    To get a feeling for how the processes were working over time, some “niches” were created by pulling apart the variables that usually go together. This was accomplished by selecting groups of students that were high versus low in perceived control at the beginning of third grade and who were in the top versus bottom quartiles of their slopes of teacher context. Then their growth curves in perceived control are plotted over time (see Figure 23.7).

    As you can see, at the beginning of third grade, the students whose teachers were supportive differed very little from students whose teachers were unsupportive. However, as students moved from grade to grade, the cumulative effect of teacher support becomes visible. Two trajectories told the typical story of positive correlations between teacher support and perceived control: Students who started high on perceived control and continuously received high levels of teacher support (which is the typical case with a positive correlation between these two variables) maintained high and steady levels of perceived control. In a similar fashion, the students who started off low in perceived control and received little teacher support over the years showed deteriorating levels of perceived control over the years. However, in these typical pathways, because student perceived control and teacher support go together, we can’t really see the effects of one versus the other.

    That’s where the beauty of the unusual case or the off-diagonals comes in. We can also describe the trajectories of perceived control for combinations of conditions that are not very common: Students with high control who do not receive teacher support and students with low control who do. In the figure, we can see the “main effects” of perceived control in that students who started low tended to stay low and students who started high tended to stay high, at least until the end of fourth grade. But starting in fifth grade, the trajectories took a different direction, with students who experienced their teachers as unsupportive are losing ground in their perceived control whereas students who experienced their teachers as supportive started gaining ground, until by the end of seventh grade the trajectories had crossed over each other.

    Do we ever get to causal inferences in these time series designs?

    It is understandable that you might think that we can never get from here to there, but there is one kind of time series design that gets us within shouting distance. It is called the “interrupted time series” or “regression discontinuity” design. You can almost guess what it looks like form the name. Researchers have many repeated measures of the potential consequence, enough that we begin to be confident about being able to predict the level and directionality of the phenomenon. Then this series is “interrupted” by the occurrence of the potential causal variable. We have our time machine, in that we think, from the past history of the consequence, that we can estimate where the consequence was headed without the interruption. And then we see whether the consequence shows an abrupt and unexpected shift after the interruption. In a regression discontinuity design, researchers test whether the slope of the consequence was changed at the point of the interruption, so that the slope before the interruption is significantly different from that after. We included a few examples of these kinds of designs in Figure 23.8 and 23.9.

    Insert Figure 23.8 and 23.9

    Such a design is considered to allow causal inference in intervention studies, where the intervention is the interruption. It is not used as often in naturalistic studies because it is harder to locate naturally-occurring “interruptions” that are both sufficiently perturbing and regular enough to allow researchers to collect their time series measures before the interruption takes place. One area in which naturally-occurring “interruptions” have been well-studied are school transitions, for example, the transition to middle school and the transition to high school. Students’ academic, motivational, and social functioning show sharp declines across these transitions, although it has taken several decades to figure out what it is about these transitions that cause declines in functioning (see box). Researchers have long been on the hunt for “natural experiments” in which unselected populations are subject to abrupt shifts in environmental factors (Campbell, 1969; Rutter, 2007).

    Transition to Middle School: Stage-Environment Fit

    What is responsible for the dramatic losses observed in students’ motivation, engagement, academic performance, and self-confidence across the transition to middle school?

    Researchers were interested in the factors that could explain these regular and significant declines in functioning. One group of researchers begin by examining the characteristics of schools that shifted from the organization of elementary schools to middle schools, things like larger schools, a variety of subject-specific teachers for shorter periods of time. Because school transitions take place across specific age ranges, a competing explanation was neurophysiological development—the idea that declines were the result of the tolls of puberty and adolescence, which would have taken place with or without a school transition.

    Very clever researchers used designs that allowed studies to separate the age changes of adolescence from the environmental transition across middle school. Researchers compared students from school districts that were organized in three different ways: (1) K-8 schools in which buildings included kindergarten through eighth grade; (2) elementary and middle schools in which districts reshuffled students from all elementary schools (K-5) into larger middle schools (6-8); and (3) elementary and junior high schools in which districts reshuffled students from all elementary schools (K-6) into larger junior high schools (7-8).

    The results of these kinds of studies were definitive (Eccles & Midgley, 1989). Adolescence was not the primary risk factor for declines in functioning—precipitous drops were apparent at whatever age the school transition took place (6th grade for districts with middle schools or 7th grade for those with junior high schools) and, most important, such drops were not seen (or were greatly reduced) across the same ages in districts that did not require school transitions (K-8 schools).

    The best account of these issues seems to be provided by stage-environment fit (Eccles et al., 1993) in which the typical changes over the transition to middle or junior high school include features (e.g., more distant and less caring teacher-student relationships, more competitive and performance-oriented learning goals, more impersonal discipline, fewer choices about academic work) that are a very bad match for the changing needs of adolescents (for stronger adult relationships outside the family, more intrinsic motivation, and greater autonomy in learning).

    Is there an elegant way to get information about both dynamics and development into one model?

    Good question. That is exactly what methods that model latent change scores (LCS) are designed to do (Ferrar & McArdle, 2010; McArdle, 2009; McArdle & Grim, 2012). These methods allow researchers to look at several things of great concern to developmentalists at the same time. Imagine a standard developmental data set that has measures of possible antecedents and outcomes at many time points. If you didn't know any better, you would look at different parts of your causal questions using all different kinds of analyses. For example, you would ask whether antecedents at one time point could predict changes in a possible outcome from that time point to the next, and then whether it could predict changes in a growth curve (i.e., launch model). Then you would go back and see if changes in the antecedent could predict changes in the outcome (change to change model) and you might lag the growth curves. Then, to examine reciprocal effects, you would go back and start at the beginning again, with your antecedents and consequences reversed. Whew!

    It was as if you could only “spend” the time that was in your design in one way at a time: It could either be used as a marker of interindividual differences in the analysis of predictive dynamics from one time to the next, or it could be used as a marker of developmental time to create a slope or growth curve. With LCS, both of these questions can be examined in the same model. For example, in our trying to answer our question about the potentially reciprocal effects of teacher involvement and student engagement, let’s imagine that we have measures of both of these variables at the beginning and the end of fifth, sixth, and seventh grades. With LCS, we can examine: (1) whether teacher involvement in fall of fifth grade predicts changes in students’ engagement across the school year; (2) whether it also does so across sixth and across seventh grades; and (3) whether those predictive effects differ. At the same time, it can examine (1) whether changes in teacher involvement predict changes in student engagement across each of those time gaps, as well as (2) whether the reciprocal pattern, that is, from student engagement to subsequent teacher involvement, also holds or is different. Of greatest interest to developmentalists, researchers can use LCS to model whether changes in teacher involvement or in engagement seem to act as leading or lagging indicators, so that one could examine whether, for example, teachers’ involvement drops as students start middle school, followed by declines in students’ engagement; or whether students’ engagement declines over the transition to middle school after which teachers begin to withdraw their support.

    Does this mean that LCS models can be used to answer all the questions that we have considered so far?

    Yes. They can be used with classic developmental designs to provide descriptive information about normative and differential trajectories, and then they can go on to chew their way through time-lagged predictions of growth curves, looking at whether they are the same or different over different developmental swaths of time. LCS models can be further enhanced through the addition of “interruptions” and hypothesized discontinuities.

    At what point are we done with “alternative plausible causal explanations”?

    In naturalistic designs? Never. The process of digging for potential causes is a never ending story. As explained by Shadish et al.

    "In quasi-experiments, the researcher has to enumerate alternative explanations one by one, decide which are plausible, and then use logic, design, and measurement to assess whether each one is operating in a way that might explain any observed effect. The difficulties are that these alternative explanations are never completely enumerable in advance, that some of them are particular to the context being studied, and that the methods needed to eliminate them from contention will vary from alternative to alternative and from study to study. (2002, p. 14)"

    That is why there is no substitute for knowledge and careful thinking about the target phenomenon, and why multiple research teams working on the same target can jostle and spur each other to examine a variety of possibilities. Some of the most useful writings on design are ones in which researchers try to outline the likely alternative explanations or threats to validity (e.g., Shadish ate al., 2002); for developmentalists, these are slightly different (e.g., Rutter, 2007; Rutter et al., 2001; Foster, 2010; Jaffee, Strait, & Odgers, 2012).

    Plausible alternative explanations for the effect.

    Given that one of Mill’s criteria for causal inference was the exclusion of other plausible alternative explanations, it is very helpful to borrow lists of such plausible alternatives, such as that provided by Michael Rutter (2007), who offers five key alternative explanatory hypotheses that should be ruled out before concluding that environmental risk factors contribute to the development of psychopathology.

    1. Genetic mediation. Risk stemming from an environmental factor is caused by genetics.

    a. Passive. Risky environments are created by adults who also pass on genetic risk.

    b. Active. People at genetic risk tend to select and create risky environments.

    2. Social selection or allocation bias. The outcome (e.g., psychopathology) is not the effect but the cause of the potential risk factor (e.g., low SES).

    3. Reverse causation. The outcome (e.g., child defiance) elicits the potential risk factor (e.g., harsh parenting).

    4. Risk feature misidentified. The risk factor is an umbrella for many components but only some of them actually cause the outcome.

    5. Confounding variables. Additional variable that both distinguishes groups to be compared and is associated with the outcome.

    a. Use of hypothesis-driven analyses to identify and test alternative pathways.

    b. Use of propensity score matching.

    c. Use of sensitivity analysis to quantify how strong a confounder must be to overturn causal inference.

    d. Use of diverse strategies and samples with different sets of potential confounders.

    e. Consider regression discontinuity to account for unmeasured confounders (despite limited applicability).