5.13: Mind in Action

Shakey was a 1960s robot that used a variety of sensors and motors to navigate through a controlled indoor environment (Nilsson, 1984). It did so by uploading its sensor readings to a central computer that stored, updated, and manipulated a model of Shakey’s world. This representation was used to develop plans of action to be put into effect, providing the important filling for Shakey’s classical sandwich.

Shakey impressed in its ability to navigate around obstacles and move objects to desired locations. However, it also demonstrated some key limitations of the classical sandwich. In particular, Shakey was extremely slow. Shakey typically required several hours to complete a task (Moravec, 1999), because the internal model of its world was computationally expensive to create and update. The problem with the sense-think-act cycle in robots like Shakey is that by the time the (slow) thinking is finished, the resulting plan may fail because the world has changed in the meantime.

The subsumption architecture of behavior-based robotics (Brooks, 1999, 2002) attempted to solve such problems by removing the classical sandwich; it was explicitly anti-representational. The logic of this radical move was that the world was its own best representation (Clark, 1997).

Behavior-based robotics took advantage of Simon’s (1969) parable of the ant, reducing costly and complex internal representations by recognizing that the external world is a critical contributor to behavior. Why expend computational resources on the creation and maintenance of an internal model of the world, when externally the world was already present, open to being sensed and to being acted upon? Classical cognitive science’s emphasis on internal representations and planning was a failure to take this parable to heart.

Interestingly, action was more important to earlier cognitive theories. Take, for example, Piaget’s theory of cognitive development (Inhelder & Piaget, 1958, 1964; Piaget, 1970a, 1970b, 1972; Piaget & Inhelder, 1969). According to this theory, in their early teens children achieve the stage of formal operations. Formal operations describe adult-level cognitive abilities that are classical in the sense that they involve logical operations on symbolic representations. Formal operations involve completely abstract thinking, where relationships between propositions are considered.

However, Piagetian theory departs from classical cognitive science by including actions in the world. The development of formal operations begins with the sensorimotor stage, which involves direct interactions with objects in the world. In the next preoperational stage these objects are internalized as symbols. The preoperational stage is followed by concrete operations. When the child is in the stage of concrete operations, symbols are manipulated, but not in the abstract: concrete operations are applied to “manipulable objects (effective or immediately imaginable manipulations), in contrast to operations bearing on propositions or simple verbal statements (logic of propositions)” (Piaget, 1972, p. 56). In short, Piaget rooted fully representational or symbolic thought (i.e., formal operations) in the child’s physical manipulation of his or her world. “The starting-point for the understanding, even of verbal concepts, is still the actions and operations of the subject” (Inhelder & Piaget, 1964, p. 284).

For example, classification and seriation (i.e., grouping and ordering entities) are operations that can be formally specified using logic or mathematics. One goal of Piagetian theory is to explain the development of such abstract competence. It does so by appealing to basic actions on the world experienced prior to the stage of formal operations, “actions which are quite elementary: putting things in piles, separating piles into lots, making alignments, and so on” (Inhelder & Piaget, 1964, p. 291).

Other theories of cognitive development share the Piagetian emphasis on the role of the world, but elaborate the notion of what aspects of the world are involved (Vygotsky, 1986). Vygotsky (1986), for example, highlighted the role of social systems—a different conceptualization of the external world—in assisting cognitive development. Vygotsky used the term zone of proximal development to define the difference between a child’s ability to solve problems without aid and their ability to solve problems when provided support or assistance. Vygotsky was strongly critical of instructional approaches that did not provide help to children as they solved problems.

Vygotsky (1986) recognized that sources of support for development were not limited to the physical world. He expanded the notion of worldly support to include social and cultural factors: “The true direction of the development of thinking is not from the individual to the social, but from the social to the individual” (p. 36). For example, to Vygotsky language was a tool for supporting cognition:

Real concepts are impossible without words, and thinking in concepts does not exist beyond verbal thinking. That is why the central moment in concept formation, and its generative cause, is a specific use of words as functional ‘tools.’ (Vygotsky, 1986, p. 107)

Clark (1997, p. 45) wrote: “We may often solve problems by ‘piggy-backing’ on reliable environmental properties. This exploitation of external structure is what I mean by the term scaffolding.” Cognitive scaffolding—the use of the world to support or extend thinking—is characteristic of theories in embodied cognitive science. Clark views scaffolding in the broad sense of a world or structure that descends from Vygotsky’s theory:

Advanced cognition depends crucially on our abilities to dissipate reasoning: to diffuse knowledge and practical wisdom through complex social structures, and to reduce the loads on individual brains by locating those brains in complex webs of linguistic, social, political, and institutional constraints. (Clark, 1997, p. 180)

While the developmental theories of Piaget and Vygotsky are departures from typical classical cognitive science in their emphasis on action and scaffolding, they are very traditional in other respects. American psychologist Sylvia Scribner pointed out that these two theorists, along with Newell and Simon, shared Aristotle’s “preoccupation with modes of thought central to theoretical inquiry—with logical operations, scientific concepts, and problem solving in symbolic domains,” maintaining “Aristotle’s high esteem for theoretical thought and disregard for the practical” (Scribner & Tobach, 1997, p. 338).

Scribner’s own work (Scribner & Tobach, 1997) was inspired by Vygotskian theory but aimed to extend its scope by examining practical cognition. Scribner described her research as the study of mind in action, because she viewed cognitive processes as being embedded with human action in the world. Scribner’s studies analyzed “the characteristics of memory and thought as they function in the larger, purposive activities which cultures organize and in which individuals engage” (p. 384). In other words, the everyday cognition studied by Scribner and her colleagues provided ample evidence of cognitive scaffolding: “Practical problem solving is an open system that includes components lying outside the formal problem— objects and information in the environment and goals and interests of the problem solver” (pp. 334–335).

One example of Scribner’s work on mind in action was the observation of problem-solving strategies exhibited by different types of workers at a dairy (Scribner & Tobach, 1997). It was discovered that a reliable difference between expert and novice dairy workers was that the former were more versatile in finding solutions to problems, largely because expert workers were much more able to exploit environmental resources. “The physical environment did not determine the problem-solving process but . . . was drawn into the process through worker initiative” (p. 377).

For example, one necessary job in the dairy was assembling orders. This involved using a computer printout of a wholesale truck driver’s order for products to deliver the next day, to fetch from different areas in the dairy the required number of cases and partial cases of various products to be loaded onto the driver’s truck. However, while the driver’s order was placed in terms of individual units (e.g., particular numbers of quarts of skim milk, of half-pints of chocolate milk, and so on), the computer printout converted these individual units into “case equivalents.” For example, one driver might require 20 quarts of skim milk. However, one case contains only 16 quarts. The computer printout for this part of the order would be 1 + 4, indicating one full case plus 4 additional units.

Scribner found differences between novice and expert product assemblers in the way in which these mixed numbers from the computer printout were converted into gathered products. Novice workers would take a purely mental arithmetic approach. As an example, consider the following protocol obtained from a novice worker:

It was one case minus six, so there’s two, four, six, eight, ten, sixteen (determines how many in a case, points finger as she counts). So there should be ten in here. Two, four, six, ten (counts units as she moves them from full to empty). One case minus six would be ten. (Scribner & Tobach, 1997, p. 302)

In contrast, expert workers were much more likely to scaffold this problem solving by working directly from the visual appearance of cases, as illustrated in a very different protocol:

I walked over and I visualized. I knew the case I was looking at had ten out of it, and I only wanted eight, so I just added two to it. I don’t never count when I’m making the order, I do it visual, a visual thing you know. (Scribner & Tobach, 1997, p. 303)

It was also found that expert workers flexibly alternated the distribution of scaffolding and mental arithmetic, but did so in a systematic way: when more mental arithmetic was employed, it was done to decrease the amount of physical exertion required to complete the order. This led to Scribner postulating a law of mental effort: “In product assembly, mental work will be expended to save physical work” (Scribner & Tobach, 1997, p. 348).

The law of mental effort was the result of Scribner’s observation that expert workers in the dairy demonstrated marked diversity and flexibility in their solutions to work-related problems. Intelligent agents may be flexible in the manner in which they allocate resources between sense-act and sense-think-act processing. Both types of processes may be in play simultaneously, but they may be applied in different amounts when the same problem is encountered at different times and under different task demands (Hutchins, 1995).

Such flexible information processing is an example of bricolage (Lévi-Strauss, 1966). A bricoleur is an “odd job man” in France.

The ‘bricoleur’ is adept at performing a large number of diverse tasks; but, unlike the engineer, he does not subordinate each of them to the availability of raw materials and tools conceived and procured for the purpose of the project. His universe of instruments is closed and the rules of his game are always to make do with ‘whatever is at hand.’ (Lévi-Strauss, 1966, p. 17)

Bricolage seems well suited to account for the flexible thinking of the sort described by Scribner. Lévi-Strauss (1966) proposed bricolage as an alternative to formal, theoretical thinking, but cast it in a negative light: “The ‘bricoleur’ is still someone who works with his hands and uses devious means compared to those of a craftsman” (pp. 16–17). Devious means are required because the bricoleur is limited to using only those components or tools that are at hand. “The engineer is always trying to make his way out of and go beyond the constraints imposed by a particular state of civilization while the ‘bricoleur’ by inclination or necessity always remains within them” (p. 19).

Recently, researchers have renewed interest in bricolage and presented it in a more positive light than did Lévi-Strauss (Papert, 1980; Turkle, 1995). To Turkle (1995), bricolage was a sort of intuition, a mental tinkering, a dialogue mediated by a virtual interface that was increasingly important with the visual GUIs of modern computing devices.

As the computer culture’s center of gravity has shifted from programming to dealing with screen simulations, the intellectual values of bricolage have become far more important.... Playing with simulation encourages people to develop the skills of the more informal soft mastery because it is so easy to run ‘What if?’ scenarios and tinker with the outcome. (Turkle, 1995, p. 52)

Papert (1980) argued that bricolage demands greater respect because it may serve as “a model for how scientifically legitimate theories are built” (p. 173).

The bricolage observed by Scribner and her colleagues when studying mind in action at the dairy revealed that practical cognition is flexibly and creatively scaffolded by an agent’s environment. However, many of the examples reported by Scribner suggest that this scaffolding involves using the environment as an external representation or memory of a problem. That the environment can be used in this fashion, as an externalized extension of memory, is not surprising. Our entire print culture—the use of handwritten notes, the writing of books—has arisen from a technology that serves as an extension of memory (McLuhan, 1994, p. 189): “Print provided a vast new memory for past writings that made a personal memory inadequate.”

However, the environment can also provide a more intricate kind of scaffolding. In addition to serving as an external store of information, it can also be exploited to manipulate its data. For instance, consider a naval navigation task in which a ship’s speed is to be computed by measuring of how far the ship has traveled over a recent interval of time (Hutchins, 1995). An internal, representational approach to performing this computation would be to calculate speed based on internalized knowledge of algebra, arithmetic, and conversions between yards and nautical miles. However, an easier external solution is possible. A navigator is much more likely to draw a line on a three-scale representation called a nomogram. The top scale of this tool indicates duration, the middle scale indicates distance, and the bottom scale indicates speed. The user marks the measured time and distance on the first two scales, joins them with a straight line, and reads the speed from the intersection of this line with the bottom scale. Thus the answer to the problem isn’t as much computed as it is inspected. “Much of the computation was done by the tool, or by its designer. The person somehow could succeed by doing less because the tool did more” (Hutchins, 1995, p. 151).

Classical cognitive science, in its championing of the representational theory of mind, demonstrates a modern persistence of the Cartesian distinction between mind and body. Its reliance on mental representation occurs at the expense of ignoring potential contributions of both an agent’s body and world. Early representational theories were strongly criticized because of their immaterial nature.

For example, consider the work of Edward Tolman (1932, 1948). Tolman appealed to representational concepts to explain behavior, such as his proposal that rats navigate and locate reinforcers by creating and manipulating a cognitive map. The mentalistic nature of Tolman’s theories was a source of harsh criticism:

Signs, in Tolman’s theory, occasion in the rat realization, or cognition, or judgment, or hypotheses, or abstraction, but they do not occasion action. In his concern with what goes on in the rat’s mind, Tolman has neglected to predict what the rat will do. So far as the theory is concerned the rat is left buried in thought; if he gets to the food-box at the end that is his concern, not the concern of the theory. (Guthrie, 1935, p. 172)

The later successes, and current dominance, of cognitive theory make such criticisms appear quaint. But classical theories are nonetheless being rigorously reformulated by embodied cognitive science.

Embodied cognitive scientists argue that classical cognitive science, with its emphasis on the disembodied mind, has failed to capture important aspects of thinking. For example, Hutchins (1995, p. 171) noted that “by failing to understand the source of the computational power in our interactions with simple ‘unintelligent’ physical devices, we position ourselves well to squander opportunities with so-called intelligent computers.” Embodied cognitive science proposes that the modern form of dualism exhibited by classical cognitive science is a mistake. For instance, Scribner hoped that her studies of mind in action conveyed “a conception of mind which is not hostage to the traditional cleavage between the mind and the hand, the mental and the manual” (Scribner & Tobach, 1997, p. 307).