In the preceding sections of this chapter we have explored the analogy that cognitive science is like classical music. This analogy was developed by comparing the characteristics of three different types of classical music to the three different schools of cognitive science: Austro-German classical music to classical cognitive science, musical Romanticism to connectionist cognitive science, and modern music to embodied cognitive science.
We also briefly reviewed how each of the three different schools has studied topics in the cognition of music. One purpose of this review was to show that each school of cognitive science has already made important contributions to this research domain. Another purpose was to show that the topics in musical cognition studied by each school reflected different, tacit views of the nature of music. For instance, the emphasis on formalism in traditional classical music is reflected in classical cognitive science’s attempt to create generative grammars of musical structure (Lerdahl & Jackendoff, 1983). Musical romanticism’s affection for the sublime is reflected in connectionist cognitive science’s use of unsupervised networks to capture regularities that cannot be formalized (Bharucha, 1999). Modern music’s rejection of the classic distinctions between composer, performer, and audience is reflected in embodied cognitive science’s exploration of how digital musical instruments can serve as behavioral objects to extend the musical mind (Bown, Eldridge, & McCormack, 2009; Magnusson, 2009).
The perspectives summarized above reflect a fragmentation of how cognitive science studies musical cognition. Different schools of cognitive science view music in dissimilar ways, and therefore they explore alternative topics using diverse methodologies. The purpose of this final section is to speculate on a different relationship between cognitive science and musical cognition, one in which the distinctions between the three different schools of thought become less important, and in which a hybrid approach to the cognition of music becomes possible.
One approach to drawing the different approaches to musical cognition together is to return to the analogy between cognitive science and classical music and to attempt to see whether the analogy itself provides room for co-operation between approaches. One of the themes of the analogy was that important differences between Austro-German music, Romantic music, and modern music existed, and that these differences paralleled those between the different schools of cognitive science. However, there are also similarities between these different types of music, and these similarities can be used to motivate commonalities between the various cognitive sciences of musical cognition. It was earlier noted that similarities existed between Austro-German classical music and musical Romanticism because the latter maintained some of the structures and traditions of the former. So let us turn instead to bridging a gap that seems much wider, the gap between Austro-German and modern music.
The differences between Austro-German classical music and modern music seem quite clear. The former is characterized by centralized control and formal structures; it is a hot medium (McLuhan, 1994) that creates marked distinctions between composer, performer, and a passive audience (Bown, Eldridge, & McCormack, 2009), and it applies the conduit metaphor (Reddy, 1979) to view the purpose of music as conveying content from composer to listener. In contrast, modern music seems to invert all of these properties. It abandons centralized control and formal structures; it is a cool medium that blurs the distinction between composer, performer, and an active audience; and it rejects the conduit metaphor and the intentional nature of music (Hanslick, 1957; Johnson, 2007).
Such dramatic differences between types of classical music suggest that it would not be surprising for very different theories to be required to explain such cognitive underpinnings. For instance, consider the task of explaining the process of musical composition. A classical theory might suffice for an account of composing Austro-German music, while a very different approach, such as embodied cognitive science, may be required to explain the composition of modern music.
One reason for considering the possibility of theoretical diversity is that in the cool medium of modern music, where control of the composition is far more decentralized, a modern piece seems more like an improvisation than a traditional composition. “A performance is essentially an interpretation of something that already exists, whereas improvisation presents us with something that only comes into being in the moment of its presentation” (Benson, 2003, p. 25). Jazz guitarist Derek Bailey (1992) noted that the ability of an audience to affect a composition is expected in improvisation: “Improvisation’s responsiveness to its environment puts the performance in a position to be directly influenced by the audience” (p. 44). Such effects, and more generally improvisation itself, are presumed to be absent from the Austro-German musical tradition: “The larger part of classical composition is closed to improvisation and, as its antithesis, it is likely that it will always remain closed” (p. 59).
However, there is a problem with this kind of dismissal. One of the shocks delivered by modern music is that many of its characteristics also apply to traditional classical music.
For instance, Austro-German music has a long tradition of improvisation, particularly in church music (Bailey, 1992). A famous example of such improvisation occurred when Johann Sebastian Bach was summoned to the court of German Emperor Frederick the Great in 1747 (Gaines, 2005). The Emperor played a theme for Bach on the piano and asked Bach to create a three-part fugue from it. The theme was a trap, probably composed by Bach’s son Carl Philipp Emanuel (employed by the Emperor), and was designed to resist the counterpoint techniques required to create a fugue. “Still, Bach managed, with almost unimaginable ingenuity, to do it, even alluding to the king’s taste by setting off his intricate counterpoint with a few gallant flourishes” (Gaines, 2005, p. 9). This was pure improvisation, as Bach composed and performed the fugue on the spot.
Benson (2003) argued that much of traditional music is actually improvisational, though perhaps less evidently than in the example above. Austro-German music was composed within the context of particular musical and cultural traditions. This provided composers with a constraining set of elements to be incorporated into new pieces, while being transformed or extended at the same time.
Composers are dependent on the ‘languages’ available to them and usually those languages are relatively well defined. What we call ‘innovation’ comes either from pushing the boundaries or from mixing elements of one language with another. (Benson, 2003, p. 43)
Benson argued that improvisation provides a better account of how traditional music is composed than do alternatives such as “creation” or “discovery,” and then showed that improvisation also applies to the performance and the reception of pre-modern works.
The example of improvisation suggests that the differences between the different traditions of classical music are quantitative, not qualitative. That is, it is not the case that Austro-German music is (for example) formal while modern music is not; instead, it may be more appropriate to claim that the former is more formal (or more centrally controlled, or less improvised, or hotter) than the latter. The possibility of quantitative distinctions raises the possibility that different types of theories can be applied to the same kind of music, and it also suggests that one approach to musical cognition may benefit by paying attention to the concerns of another.
The likelihood that one approach to musical cognition can benefit by heeding the concerns of another is easily demonstrated. For instance, it was earlier argued that musical Romanticism was reflected in connectionism’s assumption that artificial neural networks could capture regularities that cannot be formalized. One consequence of this assumption was shown to be a strong preference for the use of unsupervised networks.
However, unsupervised networks impose their own tacit restrictions upon what connectionist models can accomplish. One popular architecture used to study musical cognition is the Kohonen network (Kohonen, 1984, 2001), which assigns input patterns to winning (most-active) output units, and which in essence arranges these output units (by modifying weights) such that units that capture similar regularities are near one another in a two-dimensional map. One study that presented such a network with 115 different chords found that its output units arranged tonal centres in a pattern that reflected a noisy version of the circle of fifths (Leman, 1991).
A limitation of this kind of research is revealed by relating it to classical work on tonal organization (Krumhansl, 1990). As we saw earlier, Krumhansl found two circles of fifths (one for major keys, the other for minor keys) represented in a spiral representation wrapped around a toroidal surface. In order to capture this elegant representation, four dimensions were required (Krumhansl & Kessler, 1982). By restricting networks to representations of smaller dimensionality (such as a twodimensional Kohonen feature map), one prevents them from detecting or representing higher-dimensional regularities. In this case, knowledge gleaned from classical research could be used to explore more sophisticated network architectures (e.g., higher-dimensional self-organized maps).
Of course, connectionist research can also be used to inform classical models, particularly if one abandons “gee whiz” connectionism and interprets the internal structure of musical networks (Dawson, 2009). When supervised networks are trained on tasks involving the recognition of musical chords (Yaremchuk & Dawson, 2005; Yaremchuk & Dawson, 2008), they organize notes into hierarchies that capture circles of major seconds and circles of major thirds, as we saw in the network analyses presented in Chapter 4. As noted previously, these so-called strange circles are rarely mentioned in accounts of music theory. However, once discovered, they are just as formal and as powerful as more traditional representations such as the circle of fifths. In other words, if one ignores the sublime nature of networks and seeks to interpret their internal structures, one can discover new kinds of formal representations that could easily become part of a classical theory.
Other, more direct integrations can be made between connectionist and classical approaches to music. For example, NetNeg is a hybrid artificial intelligence system for composing two voice counterpoint pieces (Goldman et al., 1999). It assumes that some aspects of musical knowledge are subsymbolic and difficult to formalize, while other aspects are symbolic and easily described in terms of formal rules. NetNeg incorporates both types of processes to guide composition. It includes a network component that learns to reproduce melodies experienced during a training phase and uses this knowledge to generate new melodies. It also includes two rule-based agents, each of which is responsible for composing one of the voices that make up the counterpoint and for enforcing the formal rules that govern this kind of composition.
There is a loose coupling between the connectionist and the rule-based agents in NetNeg (Goldman et al., 1999), so that both co-operate, and both place constraints, on the melodies that are composed. The network suggests the next note in the melody, for either voice, and passes this information on to a rule-based agent. This suggestion, combined with interactions between the two rule-based agents (e.g., to reach an agreement on the next note to meet some aesthetic rule, such as moving the melody in opposite directions), results in each rule-based agent choosing the next note. This selection is then passed back to the connectionist part of the system to generate the next melodic prediction as the process iterates.
Integration is also possible between connectionist and embodied approaches to music. For example, for a string instrument, each note in a composition can be played by pressing different strings in different locations, and each location can be pressed by a different finger (Sayegh, 1989). The choice of string, location, and fingering is usually not specified in the composition; a performer must explore a variety of possible fingerings for playing a particular piece. Sayegh has developed a connectionist system that places various constraints on fingering so the network can suggest the optimal fingering to use. A humorous—yet strangely plausible— account of linking connectionist networks with actions was provided in Garrison Cottrell’s (1989) proposal of the “connectionist air guitar.”
Links also exist between classical and embodied approaches to musical cognition, although these are more tenuous because such research is in its infancy. For example, while Leman (2008) concentrated on the direct nature of musical experience that characterizes the embodied approach, he recognized that indirect accounts—such as verbal descriptions of music—are both common and important. The most promising links are appearing in work on the cognitive neuroscience of music, which is beginning to explore the relationship between music perception and action.
Interactions between perception of music and action have already been established. For instance, when classical music is heard, the emotion associated with it can affect perceptions of whole-body movements directed towards objects (Van den Stock et al., 2009). The cognitive neuroscience of music has revealed a great deal of evidence for the interaction between auditory and motor neural systems (Zatorre, Chen, & Penhune, 2007).
Such evidence brings to mind the notion of simulation and the role of mirror neurons, topics that were raised in Chapter 5’s discussion of embodied cognitive science. Is it possible that direct experience of musical performances engages the mirror system? Some researchers are considering this possibility (D’Ausilio, 2009; Lahav, Saltzman, & Schlaug, 2007). Lahav, Saltzman, and Schlaug (2007) trained non-musicians to play a piece of music. They then monitored their subjects’ brain activity while they listened to this newly learned piece while not performing any movements. It was discovered that motor-related areas of the brain were activated during the listening. Less activity in these areas was noted if subjects heard the same notes that were learned, but presented in a different order (i.e., as a different melody).
The mirror system has also been shown to be involved in the observation and imitation of guitar chording (Buccino et al., 2004; Vogt et al., 2007); and musical expertise, at least for professional piano players, is reflected in more specific mirror neuron processing (Haslinger et al., 2005). It has even been suggested that the mirror system is responsible for listeners misattributing anger to John Coltrane’s style of playing saxophone (Gridley & Hoff, 2006)!
A completely hybrid approach to musical cognition that includes aspects of all three schools of cognitive science is currently only a possibility. The closest realization of this possibility might be an evolutionary composing system (Todd & Werner, 1991). This system is an example of a genetic algorithm (Holland, 1992; Mitchell, 1996), which evolves a solution to a problem by evaluating the fitness of each member of a population, preserves the most fit, and then generates a new to-be-evaluated generation by combining attributes of the preserved individuals. Todd and Werner (1991) noted that such a system permits fitness to be evaluated by a number of potentially quite different critics; their model considers contributions of human, rule-based, and network critics.
Music is a complicated topic that has been considered at multiple levels of investigation, including computational or mathematical (Assayag et al., 2002; Benson, 2007; Harkleroad, 2006; Lerdahl & Jackendoff, 1983), algorithmic or behavioural (Bailey, 1992; Deutsch, 1999; Krumhansl, 1990; Seashore, 1967; Snyder, 2000), and implementational or biological (Jourdain, 1997; Levitin, 2006; Peretz & Zatorre, 2003). Music clearly is a domain that is perfectly suited to cognitive science. In this chapter, the analogy between classical music and cognitive science has been developed to highlight the very different contributions of classical, connectionist, and embodied cognitive science to the study of musical cognition. It raised the possibility of a more unified approach to musical cognition that combines elements of all three different schools of thought.