Skip to main content
Social Sci LibreTexts

3.14: The Impenetrable Architecture

  • Page ID
    35725
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    Classical cognitive scientists often develop theories in the form of working computer simulations. These models are validated by collecting evidence that shows they are strongly equivalent to the subjects or phenomena being modelled. This begins by first demonstrating weak equivalence, that both model and subject are computing the same input-output function. The quest for strong equivalence is furthered by using intermediate state evidence, relative complexity evidence, and error evidence to demonstrate, in striking detail, that both model and subject are employing the same algorithm.

    However, strong equivalence can only be established by demonstrating an additional relationship between model and subject. Not only must model and subject be employing the same algorithm, but both must also be employing the same primitive processes. Strong equivalence requires architectural equivalence.

    The primitives of a computer simulation are readily identifiable. A computer simulation should be a collection of primitives that are designed to generate a behaviour of interest (Dawson, 2004). In order to create a model of cognition, one must define the basic properties of a symbolic structure, the nature of the processes that can manipulate these expressions, and the control system that chooses when to apply a particular rule, operation, or process. A model makes these primitive characteristics explicit. When the model is run, its behaviour shows what these primitives can produce.

    While identifying a model’s primitives should be straightforward, determining the architecture employed by a modelled subject is far from easy. To illustrate this, let us consider research on mental imagery.

    Mental imagery is a cognitive phenomenon in which we experience or imagine mental pictures. Mental imagery is often involved in solving spatial problems (Kosslyn, 1980). For instance, imagine being asked how many windows there are on the front wall of the building in which you live. A common approach to answering this question would be to imagine the image of this wall and to inspect the image, mentally counting the number of windows that are displayed in it. Mental imagery is also crucially important for human memory (Paivio, 1969, 1971, 1986; Yates, 1966): we are better at remembering items if we can create a mental image of them. Indeed, the construction of bizarre mental images, or of images that link two or more items together, is a standard tool of the mnemonic trade (Lorayne, 1985, 1998, 2007; Lorayne & Lucas, 1974).

    An early achievement of the cognitive revolution in psychology (Miller, 2003; Vauclair & Perret, 2003) was a rekindled interest in studying mental imagery, an area that had been neglected during the reign of behaviourism (Paivio, 1971, 1986). In the early stages of renewed imagery research, traditional paradigms were modified to solidly establish that concept imageability was a key predictor of verbal behaviour and associative learning (Paivio, 1969). In later stages, new paradigms were invented to permit researchers to investigate the underlying nature of mental images (Kosslyn, 1980; Shepard & Cooper, 1982).

    For example, consider the relative complexity evidence obtained using the mental rotation task (Cooper & Shepard, 1973a, 1973b; Shepard & Metzler, 1971). In this task, subjects are presented with a pair of images. In some instances, the two images are of the same object. In other instances, the two images are different (e.g., one is a mirror image of the other). The orientation of the images can also be varied—for instance, they can be rotated to different degrees in the plane of view. The angular disparity between the two images is the key independent variable. A subject’s task is to judge whether the images are the same or not; the key dependent measure is the amount of time required to respond.

    In order to perform the mental rotation task, subjects first construct a mental image of one of the objects, and then imagine rotating it to the correct orientation to enable them to judge whether it is the same as the other object. The standard finding in this task is that there is a linear relationship between response latency and the amount of mental rotation that is required. From these results it has been concluded that “the process of mental rotation is an analog one in that intermediate states in the process have a one-to-one correspondence with intermediate stages in the external rotation of an object” (Shepard & Cooper, 1982, p. 185). That is, mental processes rotate mental images in a holistic fashion, through intermediate orientations, just as physical processes can rotate real objects.

    Another source of relative complexity evidence concerning mental imagery is the image scanning task (Kosslyn, 1980; Kosslyn, Ball, & Reisler, 1978). In the most famous version of this task, subjects are first trained to create an accurate mental image of an island map on which seven different locations are marked. Then subjects are asked to construct this mental image, focusing their attention at one of the locations. They are then provided with a name, which may or may not be one of the other map locations. If the name is of another map location, then subjects are instructed to scan across the image to it, pressing a button when they arrive at the second location.

    In the map-scanning version of the image-scanning task, the dependent variable was the amount of time from the naming of the second location to a subject’s button press, and the independent variable was the distance on the map between the first and second locations. The key finding was that there was nearly a perfectly linear relationship between latency and distance (Kosslyn Ball, & Reisler, 1978): an increased distance led to an increased response latency, suggesting that the image had spatial extent, and that it was scanned at a constant rate.

    The scanning experiments support the claim that portions of images depict corresponding portions of the represented objects, and that the spatial relations between portions of the image index the spatial relations between the corresponding portions of the imaged objects. (Kosslyn, 1980, p. 51)

    The relative complexity evidence obtained from tasks like mental rotation and image scanning provided the basis for a prominent account of mental imagery known as the depictive theory (Kosslyn, 1980, 1994; Kosslyn, Thompson, & Ganis, 2006). This theory is based on the claim that mental images are not merely internal representations that describe visuospatial information (as would be the case with words or with logical propositions), but instead depict this information because the format of an image is quasi-pictorial. That is, while a mental image is not claimed to literally be a picture in the head, it nevertheless represents content by resemblance.

    There is a correspondence between parts and spatial relations of the representation and those of the object; this structural mapping, which confers a type of resemblance, underlies the way images convey specific content. In this respect images are like pictures. Unlike words and symbols, depictions are not arbitrarily paired with what they represent. (Kosslyn, Thompson, & Ganis, 2006, p. 44)

    The depictive theory specifies primitive properties of mental images, which have sometimes been called privileged properties (Kosslyn, 1980). What are these primitives? One is that images occur in a spatial medium that is functionally equivalent to a coordinate space. A second is that images are patterns that are produced by activating local regions of this space to produce an “abstract spatial isomorphism” (Kosslyn, 1980, p. 33) between the image and what it represents. This isomorphism is a correspondence between an image and a represented object in terms of their parts as well as spatial relations amongst these parts. A third is that images not only depict spatial extent, they also depict properties of visible surfaces such as colour and texture.

    These privileged properties are characteristic of the format mental images—the structure of images as symbolic expressions. When such a structure is paired with particular primitive processes, certain types of questions are easily answered. These processes are visual in nature: for instance, mental images can be scanned, inspected at different apparent sizes, or rotated. The coupling of such processes with the depictive structure of images is well-suited to solving visuospatial problems. Other structure-process pairings—in particular, logical operations on propositional expressions that describe spatial properties (Pylyshyn, 1973)—do not make spatial information explicit and arguably will not be as adept at solving visuospatial problems. Kosslyn (1980, p. 35) called the structural properties of images privileged because their possession “[distinguishes] an image from other forms of representation.”

    That the depictive theory makes claims about the primitive properties of mental images indicates quite clearly that it is an account of cognitive architecture. That it is a theory about architecture is further supported by the fact that the latest phase of imagery research has involved the supplementing behavioural data with evidence concerning the cognitive neuroscience of imagery (Kosslyn, 1994; Kosslyn et al., 1995; Kosslyn et al., 1999; Kosslyn, Thompson, & Alpert, 1997; Kosslyn, Thompson, & Ganis, 2006). This research has attempted to ground the architectural properties of images into topographically organized regions of the cortex.

    Computer simulation has proven to be a key medium for evaluating the depictive theory of mental imagery. Beginning with work in the late 1970s (Kosslyn & Shwartz, 1977), the privileged properties of mental images have been converted into a working computer model (Kosslyn, 1980, 1987, 1994; Kosslyn et al., 1984; Kosslyn et al., 1985). In general terms, over time these models represent an elaboration of a general theoretical structure: long-term memory uses propositional structures to store spatial information. Image construction processes convert this propositional information into depictive representations on a spatial medium that enforces the primitive structural properties of images. Separate from this medium are primitive processes that operate on the depicted information (e.g., scan, inspect, interpret). This form of model has shown that the privileged properties of images that define the depictive theory are sufficient for simulating a wide variety of the regularities that govern mental imagery.

    The last few paragraphs have introduced Kosslyn’s (e.g., 1980) depictive theory, its proposals about the privileged properties of mental images, and the success that computer simulations derived from this theory have had at modelling behavioural results. All of these topics concern statements about primitives in the domain of a theory or model about mental imagery. Let us now turn to one issue that has not yet been addressed: the nature of the primitives employed by the modelled subject, the human imager.

    The status of privileged properties espoused by the depictive theory has been the subject of a decades-long imagery debate (Block, 1981; Tye, 1991). At the heart of the imagery debate is a basic question: are the privileged properties parts of the architecture or not? The imagery debate began with the publication of a seminal paper (Pylyshyn, 1973), which proposed that the primitive properties of images were not depictive, but were instead descriptive properties based on a logical or propositional representation. This position represents the basic claim of the propositional theory, which stands as a critical alternative to the depictive theory.

    The imagery debate continues to the present day; propositional theory’s criticism of the depictive position has been prolific and influential (Pylyshyn, 1981a, 1981b, 1984, 2003a, 2003b, 2003c, 2007). The imagery debate has been contentious, has involved a number of different subtle theoretical arguments about the relationship between theory and data, and has shown no signs of being clearly resolved. Indeed, some have argued that it is a debate that is cannot be resolved, because it is impossible to identify data that is appropriate to differentiate the depictive and propositional theories (Anderson, 1978). In this section, the overall status of the imagery debate is not of concern. We are instead interested in a particular type of evidence that has played an important role in the debate: evidence concerning cognitive penetrability (Pylyshyn, 1980, 1984, 1999).

    Recall from the earlier discussion of algorithms and architecture that Newell (1990) proposed that the rate of change of various parts of a physical symbol system would differ radically depending upon which component was being examined. Newell observed that data should change rapidly, stored programs should be more enduring, and the architecture that interprets stored programs should be even more stable. This is because the architecture is wired in. It may change slowly (e.g., in human cognition because of biological development), but it should be the most stable information processing component. When someone claims that they have changed their mind, we interpret this as meaning that they have updated their facts, or that they have used a new approach or strategy to arrive at a conclusion. We don’t interpret this as a claim that they have altered their basic mental machinery—when we change our mind, we don’t change our cognitive architecture!

    The cognitive penetrability criterion (Pylyshyn, 1980, 1984, 1999) is an experimental paradigm that takes advantage of the persistent “wired in” nature of the architecture. If some function is part of the architecture, then it should not be affected by changes in cognitive content—changing beliefs should not result in a changing architecture. The architecture is cognitively impenetrable. In contrast, if some function changes because of a change in content that is semantically related to the function, then this is evidence that it is not part of the architecture.

    If a system is cognitively penetrable then the function it computes is sensitive, in a semantically coherent way, to the organism’s goals and beliefs, that is, it can be altered in a way that bears some logical relation to what the person knows. (Pylyshyn, 1999, p. 343)

    The architecture is not cognitively penetrable.

    Cognitive penetrability provides a paradigm for testing whether a function of interest is part of the architecture or not. First, some function is measured as part of a pre-test. For example, consider Figure 3-13, which presents the Müller-Lyer illusion, which was discovered in 1889 (Gregory, 1978). In a pre-test, it would be determined whether you experience this illusion. Some measurement would be made to determine whether you judge the horizontal line segment of the top arrow to be longer than the horizontal line segment of the bottom arrow.

    Second, a strong manipulation of a belief related to the function that produces the Müller-Lyer illusion would be performed. You, as a subject, might be told that the two horizontal line segments were equal in length. You might be given a ruler, and asked to measure the two line segments, in order to convince yourself that your experience was incorrect and that the two lines were of the same length.

    clipboard_ed4349392340e9d07d4589830e28ffb80.png

    Figure 3-13. The Müller-Lyer illusion.

    Third, a post-test would determine whether you still experienced the illusion. Do the line segments still appear to be of different length, even though you are armed with the knowledge that this appearance is false? This illusion has had such a long history because its appearance is not affected by such cognitive content. The mechanism that is responsible for the Müller-Lyer illusion is cognitively impenetrable.

    This paradigm has been applied to some of the standard mental imagery tasks in order to show that some of the privileged properties of images are cognitively penetrable and therefore cannot be part of the architecture. For instance, in his 1981 dissertation, Liam Bannon examined the map scanning task for cognitive penetrability (for methodological details, see Pylyshyn, 1981a). Bannon reasoned that the instructions given to subjects in the standard map scanning study (Kosslyn,Ball, & Reiser, 1978) instilled a belief that image scanning was like scanning a picture. Bannon was able to replicate the Kosslyn, Ball, & Reiser results in one condition. However, in other conditions the instructions were changed so that the images had to be scanned to answer a question, but no beliefs about scanning were instilled. In one study, Bannon had subjects shift attention from the first map location to the second (named) location, and then judge the compass direction from the second location to the first. In this condition, the linearly increasing relationship between distance and time disappeared. Image scanning appears to be cognitively penetrable, challenging some of the architectural claims of depictive theory. “Images can be examined without the putative constraints of the surface display postulated by Kosslyn and others” (Pylyshyn, 1981a, p. 40).

    The cognitive penetrability paradigm has also been applied to the mental rotation task (Pylyshyn, 1979b). Pylyshyn reasoned that if mental rotation is accomplished by primitive mechanisms, then it must be cognitively impenetrable. One prediction that follows from this reasoning is that the rate of mental rotation should be independent of the content being rotated—an image depicting simple content should, by virtue of its putative architectural nature, be rotated at the same rate as a different image depicting more complex content.

    Pylyshyn (1979b) tested this hypothesis in two experiments and found evidence of cognitive penetration. The rate of mental rotation was affected by practice, by the content of the image being rotated, and by the nature of the comparison task that subjects were asked to perform. As was the case with image scanning, it would seem that the “analog” rotation of images is not primitive, but is instead based on simpler processes that do belong to the architecture.

    The more carefully we examine phenomena, such as the mental rotation findings, the more we find that the informally appealing holistic image-manipulation views must be replaced by finer grained piecemeal procedures that operate upon an analyzed and structured stimulus using largely serial, resource-limited mechanisms. (Pylyshyn, 1979b, p. 27)

    Cognitive penetrability has played an important role in domains other than mental imagery. For instance, in the literature concerned with social perception and prediction, there is debate between a classical theory called theory-theory (Gopnik & Meltzoff, 1997; Gopnik &Wellman, 1992) and a newer approach called simulation theory (Gordon, 1986, 2005b), which is nicely situated in the embodied cognitive science that is the topic of Chapter 5. There is a growing discussion about whether cognitive penetrability can be used to discriminate between these two theories (Greenwood, 1999; Heal, 1996; Kuhberger et al., 2006; Perner et al., 1999; Stich &Nichols, 1997). Cognitive penetrability has also been applied to various topics in visual perception (Raftopoulos, 2001), including face perception (Bentin & Golland, 2002) and the perception of illusory motion (Dawson, 1991; Dawson &Wright, 1989; Wright & Dawson, 1994).

    While cognitive penetrability is an important tool when faced with the challenge of examining the architectural equivalence between model and subject, it is not without its problems. For instance, in spite of it being applied to the study of mental imagery, the imager debate rages on, suggesting that penetrability evidence is not as compelling or powerful as its proponents might hope. Perhaps one reason for this is that it seeks a null result—the absence of an effect of cognitive content on cognitive function. While cognitive penetrability can provide architectural evidence for strong equivalence, other sources of evidence are likely required. One source of such additional evidence is cognitive neuroscience.


    This page titled 3.14: The Impenetrable Architecture is shared under a CC BY-NC-ND license and was authored, remixed, and/or curated by Michael R. W. Dawson (Athabasca University Press) .

    • Was this article helpful?