The goal of cognitive science is to explain cognitive phenomena. One approach to such explanation is to generate a set of laws or principles that capture the regularities that are exhibited by members that belong to a particular class. Once it is determined that some new system belongs to a class, then it is expected that the principles known to govern that class will also apply to the new system. In this sense, the laws governing a class capture generalizations (Pylyshyn, 1984).
The problem that faced cognitive science in its infancy was that the classes of interest, and the laws that captured generalizations about their members, depended upon which level of analysis was adopted (Marr, 1982). For instance, at a physical level of investigation, electromechanical and digital computers do not belong to the same class. However, at a more abstract level of investigation (e.g., at the architectural level described in Chapter 2), these two very different types of physical devices belong to the same class, because their components are functionally equivalent: “Many of the electronic circuits which performed the basic arithmetic operations [in ENIAC] were simply electronic analogs of the same units used in mechanical calculators and the commercial accounting machines of the day” (Williams, 1997, p. 272).
The realization that cognitive systems must be examined from multiple levels of analysis motivated Marr’s (1982) tri-level hypothesis. According to this hypothesis, cognitive systems must be explained at three different levels of analysis: physical, algorithmic, and computational.
It is not enough to be able to predict locally the responses of single cells, nor is it enough to be able to predict locally the results of psychophysical experiments. Nor is it enough to be able to write computer programs that perform approximately in the desired way. One has to do all these things at once and also be very aware of the additional level of explanation that I have called the level of computational theory. (Marr, 1982, pp. 329–330)
The tri-level hypothesis provides a foundation for cognitive science and accounts for its interdisciplinary nature (Dawson, 1998). This is because each level of analysis uses a qualitatively different vocabulary to ask questions about cognitive systems and uses very different methods to provide the answers to these questions. That is, each level of analysis appeals to the different languages and techniques of distinct scientific disciplines. The need to explain cognitive systems at different levels of analysis forces cognitive scientists to be interdisciplinary.
Marr’s (1982) tri-level hypothesis can also be used to compare the different approaches to cognitive science. Is the tri-level hypothesis equally applicable to the three different schools of thought? Provided that the three levels are interpreted at a moderately coarse level, it would appear that this question could be answered affirmatively.
At Marr’s (1982) implementational level, cognitive scientists ask how information processes are physically realized. For a cognitive science of biological agents, answers to implementational-level questions are phrased in a vocabulary that describes biological mechanisms. It would appear that all three approaches to cognitive science are materialist and as a result are interested in conducting implementational-level analyses. Differences between the three schools of thought at this level might only be reflected in the scope of biological mechanisms that are of interest. In particular, classical and connectionist cognitive scientists will emphasize neural mechanisms, while embodied cognitive scientists are likely to be interested not only in the brain but also in other parts of the body that interact with the external world.
At Marr’s (1982) algorithmic level, cognitive scientists are interested in specifying the procedures that are used to solve particular information processing problems. At this level, there are substantial technical differences amongst the three schools of thought. For example, classical and connectionist cognitive scientists would appeal to very different kinds of representations in their algorithmic accounts (Broadbent, 1985; Rumelhart & McClelland, 1985). Similarly, an algorithmic account of internal planning would be quite different from an embodied account of controlled action, or of scaffolded, cognition. In spite of such technical differences, though, it would be difficult to claim that one approach to cognitive science provides procedural accounts, while another does not. All three approaches to cognitive science are motivated to investigate at the algorithmic level.
At Marr’s (1982) computational level, cognitive scientists wish to determine the nature of the information processing problems being solved by agents. Answering these questions usually requires developing proofs in some formal language. Again, all three approaches to cognitive science are well versed in posing computationallevel questions. The differences between them are reflected in the formal language used to explore answers to these questions. Classical cognitive science often appeals to some form of propositional logic (Chomsky, 1959a; McCawley, 1981; Wexler & Culicover, 1980), the behaviour of connectionist networks lends itself to being described in terms of statistical mechanics (Amit, 1989; Grossberg, 1988; Smolensky, 1988; Smolensky & Legendre, 2006), and embodied cognitive scientists have a preference for dynamical systems theory (Clark, 1997; Port & van Gelder, 1995b; Shapiro, 2011).
Marr’s (1982) tri-level hypothesis is only one example of exploring cognition at multiple levels. Precursors of Marr’s approach can be found in core writings that appeared fairly early in cognitive science’s modern history. For instance, philosopher Jerry Fodor (1968b) noted that one cannot establish any kind of equivalence between the behaviour of an organism and the behaviour of a simulation without first specifying a level of description that places the comparison in a particular context.
Marr (1982) himself noted that an even stronger parallel exists between the tri-level hypothesis and Chomsky’s (1965) approach to language. To begin with, Chomsky’s notion of an innate and universal grammar, as well as his idea of a “language organ” or a “faculty of language,” reflect a materialist view of language. Chomsky clearly expects that language can be investigated at the implementational level. The language faculty is due “to millions of years of evolution or to principles of neural organization that may be even more deeply grounded in physical law” (p. 59). Similarly, “the study of innate mechanisms leads us to universal grammar, but also, of course, to investigation of the biologically determined principles that underlie language use” (Chomsky, 1980, p. 206).
Marr’s (1982) algorithmic level is mirrored by Chomsky’s (1965) concept of linguistic performance. Linguistic performance is algorithmic in the sense that a performance theory should account for “the actual use of language in concrete situations” (Chomsky, 1965, p. 4). The psychology of language can be construed as being primarily concerned with providing theories of performance (Chomsky, 1980). That is, psychology’s “concern is the processes of production, interpretation, and the like, which make use of the knowledge attained, and the processes by which transition takes place from the initial to the final state, that is, language acquisition” (pp. 201– 202). An account of the processes that underlie performance requires an investigation at the algorithmic level.
Finally, Marr (1982) noted that Chomsky’s notion of linguistic competence parallels the computational level of analysis. A theory of linguistic competence specifies an ideal speaker-listener’s knowledge of language (Chomsky, 1965). A grammar is a theory of competence; it provides an account of the nature of language that “is unaffected by such grammatically irrelevant conditions as memory limitations, distractions, shifts of attention and interest, and errors (random or characteristic) in applying . . . knowledge of the language in actual performance” (p. 3). As a computational-level theory, a grammar accounts for what in principle could be said or understood; in contrast, a performance theory accounts for language behaviours that actually occurred (Fodor, 1968b). Marr (1982) argued that influential criticisms of Chomsky’s theory (Winograd, 1972a) mistakenly viewed transformational grammar as an algorithmic, and not a computational, account. “Chomsky’s theory of transformational grammar is a true computational theory . . . concerned solely with specifying what the syntactic decomposition of an English sentence should be, and not at all with how that decomposition should be achieved” (Marr, 1982, p. 28).
The notion of the cognitive vocabulary arises by taking a different approach to linking Marr’s (1982) theory of vision to Chomsky’s (1965) theory of language. In addition to proposing the tri-level hypothesis, Marr detailed a sequence of different types of representations of visual information. In the early stages of visual processing, information was represented in the primal sketch, which provided a spatial representation of visual primitives such as boundaries between surfaces. Operations on the primal sketch produced the 2½-D sketch, which represents the properties, including depth, of all visible surfaces. Finally, operations on the 2½-D sketch produce the 3-D model, which represents the three-dimensional properties of objects (including surfaces not directly visible) in a fashion that is independent of view.
Chomsky’s (1965) approach to language also posits different kinds of representations (Jackendoff, 1987). These include representations of phonological structure, representations of syntax, and representations of semantic or conceptual structures. Jackendoff argued that Marr’s (1982) theory of vision could be directly linked to Chomsky’s theory of language by a mapping between 3-D models and conceptual structures. This link permits the output of visual processing to play a critical role in fixing the semantic content of linguistic representations (Jackendoff, 1983, 1990).
One key element of Jackendoff’s (1987) proposal is the distinction that he imposed between syntax and semantics. This type of separation is characteristic of classical cognitive science, which strives to separate the formal properties of symbols from their content-bearing properties (Haugeland, 1985).
For instance, classical theorists define symbols as physical patterns that bear meaning because they denote or designate circumstances in the real world (Vera & Simon, 1993). The physical pattern part of this definition permits symbols to be manipulated in terms of their shape or form: all that is required is that the physical nature of a pattern be sufficient to identify it as a token of some symbolic type. The designation aspect of this definition concerns the meaning or semantic content of the symbol and is completely separate from its formal or syntactic nature.
To put it dramatically, interpreted formal tokens lead two lives: SYNTACTICAL LIVES, in which they are meaningless markers, moved according to the rules of some self-contained game; and SEMANTIC LIVES, in which they have meanings and symbolic relations to the outside world. (Haugeland, 1985, p. 100)
In other words, when cognitive systems are viewed representationally (e.g., as in Jackendoff, 1987), they can be described at different levels, but these levels are not identical to those of Marr’s (1982) tri-level hypothesis. Representationally, one level is physical, involving the physical properties of symbols. A second level is formal, concerning the logical properties of symbols. A third level is semantic, regarding the meanings designated by symbols. Again, each of these levels involves using a particular vocabulary to capture its particular regularities.
This second sense of levels of description leads to a position that some researchers have used to distinguish classical cognitive science from other approaches. In particular, it is first proposed that a cognitive vocabulary is used to capture regularities at the semantic level of description. It is then argued that the cognitive vocabulary is a mark of the classical, because it is a vocabulary that is used by classical cognitive scientists, but which is not employed by their connectionist or embodied counterparts.
The cognitive vocabulary is used to capture regularities at the cognitive level that cannot be captured at the physical or symbolic levels (Pylyshyn, 1984). “But what sort of regularities can these be? The answer has already been given: precisely the regularities that tie goals, beliefs, and actions together in a rational manner” (p. 132). In other words, the cognitive vocabulary captures regularities by describing meaningful (i.e., rational) relations between the contents of mental representations. It is the vocabulary used when one adopts the intentional stance (Dennett, 1987) to predict future behaviour or when one explains an agent at the knowledge level (Newell, 1982, 1993).
To treat a system at the knowledge level is to treat it as having some knowledge and some goals, and believing it will do whatever is within its power to attain its goals, in so far as its knowledge indicates. (Newell, 1982, p. 98)
The power of the cognitive vocabulary is that it uses meaningful relations between mental contents to explain intelligent behaviour (Fodor & Pylyshyn, 1988). For instance, meaningful, complex tokens are possible because the semantics of such expressions are defined in terms of the contents of their constituent symbols as well as the structural relationships that hold between these constituents. The cognitive vocabulary’s exploitation of constituent structure leads to the systematicity of classical theories: if one can process some expressions, then it is guaranteed that other expressions can also be processed because of the nature of constituent structures. This in turn permits classical theories to be productive, capable of generating an infinite variety of expressions from finite resources.
Some classical theorists have argued that other approaches in cognitive science do not posit the structural relations between mental contents that are captured by the cognitive vocabulary (Fodor & Pylyshyn, 1988). For instance, Fodor and Pylyshyn (1988) claimed that even though connectionist theories are representational, they are not cognitive because they exploit a very limited kind of relationship between represented contents.
Classical theories disagree with Connectionist theories about what primitive relations hold among these content-bearing entities. Connectionist theories acknowledge only causal connectedness as a principled relation among nodes; when you know how activation and inhibition flow among them, you know everything there is to know about how the nodes in a network are related. (Fodor and Pylyshyn, 1988, p. 12)
As a result, Fodor and Pylyshyn argued, connectionist models are not componential, nor systematic, nor even productive. In fact, because they do not use a cognitive vocabulary (in the full classical sense), connectionism is not cognitive.
Related arguments can be made against positions that have played a central role in embodied cognitive science, such as the ecological approach to perception advocated by Gibson (1979). Fodor and Pylyshyn (1981) have argued against the notion of direct perception, which attempts to construe perception as involving the direct pick-up of information about the layout of a scene; that is, acquiring this information without the use of inferences from cognitive contents: “The fundamental difficulty for Gibson is that ‘about’ (as in ‘information about the layout in the light’) is a semantic relation, and Gibson has no account at all of what it is to recognize a semantic relation” (p. 168). Fodor and Pylyshyn argued that Gibson’s only notion of information involves the correlation between states of affairs, and that this notion is insufficient because it is not as powerful as the classical notion of structural relations among cognitive contents. “The semantic notion of information that Gibson needs depends, so far as anyone knows, on precisely the mental representation construct that he deplores” (p. 168).
It is clear from the discussion above that Pylyshyn used the cognitive vocabulary to distinguish classical models from connectionist and embodied theories. This does not mean that he believed that non-classical approaches have no contributions to make. For instance, in Chapter 8 we consider in detail his theory of seeing and visualizing (Pylyshyn, 2003c, 2007); it is argued that this is a hybrid theory, because it incorporates elements from all three schools of thought in cognitive science.
However, one of the key elements of Pylyshyn’s theory is that vision is quite distinct from cognition; he has made an extended argument for this position. When he appealed to connectionist networks or embodied access to the world, he did so in his account of visual, and not cognitive, processes. His view has been that such processes can only be involved in vision, because they do not appeal to the cognitive vocabulary and therefore cannot be viewed as cognitive processes. In short, the cognitive vocabulary is viewed by Pylyshyn as a mark of the classical.
Is the cognitive vocabulary a mark of the classical? It could be—provided that the semantic level of explanation captures regularities that cannot be expressed at either the physical or symbolic levels. Pylyshyn (1984) argued that this is indeed the case, and that the three different levels are independent:
The reason we need to postulate representational content for functional states is to explain the existence of certain distinctions, constraints, and regularities in the behavior of at least human cognitive systems, which, in turn, appear to be expressible only in terms of the semantic content of the functional states of these systems. Chief among the constraints is some principle of rationality. (Pylyshyn, 1984, p. 38)
However, it is not at all clear that in the practice of classical cognitive science—particularly the development of computer simulation models—the cognitive level is distinct from the symbolic level. Instead, classical researchers adhere to what is known as the formalist’s motto (Haugeland, 1985). That is, the semantic regularities of a classical model emerge from the truth-preserving, but syntactic, regularities at the symbolic level.
If the formal (syntactical) rules specify the relevant texts and if the (semantic) interpretation must make sense of all those texts, then simply playing by the rules is itself a surefire way to make sense. Obey the formal rules of arithmetic, for instance, and your answers are sure to be true. (Haugeland, 1985, p. 106)
If this relation holds between syntax and semantics, then the cognitive vocabulary is not capturing regularities that cannot be captured at the symbolic level.
The formalist’s motto is a consequence of the physical symbol system hypothesis (Newell, 1980; Newell & Simon, 1976) that permitted classical cognitive science to replace Cartesian dualism with materialism. Fodor and Pylyshyn (1988, p. 13) adopt the physical symbol system hypothesis, and tacitly accept the formalist’s motto: “Because Classical mental representations have combinatorial structure, it is possible for Classical mental operations to apply to them by reference to their form.” Note that in this quote, operations are concerned with formal and not semantic properties; semantics is preserved provided that there is a special relationship between constraints on symbol manipulations and constraints on symbolic content.
To summarize this section: The interdisciplinary nature of cognitive science arises because cognitive systems require explanations at multiple levels. Two multiple level approaches are commonly found in the cognitive science literature. The first is Marr’s (1982) tri-level hypothesis, which requires cognitive systems to be explained at the implementational, algorithmic, and computational levels. It is argued above that all three schools of thought in cognitive science adhere to the tri-level hypothesis. Though at each level there are technical differences to be found between classical, connectionist, and embodied cognitive science, all three approaches seem consistent with Marr’s approach. The tri-level hypothesis cannot be used to distinguish one cognitive science from another.
The second is a tri-level approach that emerges from the physical symbol system hypothesis. It argues that information processing requires explanation at three independent levels: the physical, the symbolic, and the semantic (Dennett, 1987; Newell, 1982; Pylyshyn, 1984). The physical and symbolic levels in this approach bear a fairly strong relationship to Marr’s (1982) implementational and algorithmic levels. The semantic level, though, differs from Marr’s computational level in calling for a cognitive vocabulary that captures regularities by appealing to the contents of mental representations. This cognitive vocabulary has been proposed as a mark of the classical that distinguishes classical theories from those proposed by connectionist and embodied researchers. However, it has been suggested that this view may not hold, because the formalist’s motto makes the proposal of an independent cognitive vocabulary difficult to defend.