Dual Coding and Common Coding Theories of Memory

The Dual Coding Theory of memory was initially proposed Paivio (1971) in order to explain the powerful mnemonic effects of imagery that he and others had uncovered, but its implications for cognitive theory go far beyond these findings. It has inspired an enormous amount of controversy and experimental research in psychology, and played a very large role in stimulating the resurgence of scientific and philosophical interest in imagery. Indeed, it has been described as “one of the most influential theories of cognition this [20th] century” (Marks, 1997), and has been fruitfully applied to a wide range of psychological issues, including: thinking processes (Paivio, 1975a); individual differences in thinking styles (Paivio & Harshman, 1983); language understanding (Paivio & Begg, 1981); bilingualism (Paivio & Desrochers, 1980; Paivio, 1986); metaphor (Paivio, 1979); creative thinking (Paivio, 1983b); the observational/theoretical distinction in science (Clark & Paivio, 1989); the psychology of reading and writing (Sadoski & Paivio, 2001); and even the use of visualization to enhance athletic performance (Munroe et al., 2000).

The more intricate details of Dual Coding Theory are beyond our scope here, but the core idea is very simple and intuitive. Paivio proposes that the human mind operates with two distinct classes of mental representation (or “codes”), verbal representations and mental images, and that human memory thus comprises two functionally independent (although interacting) systems or stores, verbal memory and image memory. Imagery potentiates recall of verbal material because when a word evokes an associated image (either spontaneously, or through deliberate effort) two separate but linked memory traces are laid down, one in each of the memory stores. Obviously the chances that a memory will be retained and retrieved are much greater if it is stored in two distinct functional locations rather than in just one.

The laboratory evidence favoring the theory goes well beyond the original context of verbal learning experiments, however. For example, it is claimed that it finds experimental support in studies of memory for pictures (Richardson, 1980 ch.5) and in chronometric studies of mental comparisons of sizes, distances and other dimensions of variation (Paivio, 1975, 1978a, 1978b; Kosslyn, Murphy, Bemesderfer, & Feinstein, 1977; Moyer & Dumais, 1978). Perhaps the most direct experimental support comes from work on the so called selective interference effect, which occurs when a person tries simultaneously to do two mental tasks both of which call for manipulation of representations from the same code (i.e., two verbal tasks, or two imagery/visuo-spatial tasks). In such circumstances, experimental subjects perform measurably more poorly (i.e., slower and/or with more errors) than they do when attempting either task together with one that calls upon the other code (i.e., a verbal and a simultaneous imagery/visuo-spatial task) (Brooks, 1967, 1968; Atwood, 1971; Segal & Fusella, 1971; Baddeley et al., 1975; Janssen, 1976a, 1976b; Baddeley & Lieberman, 1980; Eddy & Glass, 1981; Hampson & Duffy, 1984; Logie & Baddeley, 1990; De Beni & Moè, 2003). Provided one interprets the imagery code primarily as a system for the representation of shape and spatial and spatio-temporal relationships (rather than as specialized for encoding purely visual properties such as color or brightness) these results find a very natural interpretation in Dual Coding terms: two tasks that use the same code interfere strongly with one another because they call upon the same representational and processing resources.^[1]

Nevertheless, the proper interpretation of these experiments (and all the other multifarious relevant laboratory findings) and thus the empirical status of Dual Coding Theory itself, remains controversial. Throughout its history, the theory has been developed and interpreted in the context of opposition to various forms of what have come to be known as common coding theories of memory: Theories committed to explaining all the relevant phenomena in terms of just one type of code (representational format) common to all memories. Thus, to properly understand Dual Coding Theory and the controversies that have surrounded it, it is important to be aware that the prevailing conception of the nature of this common code have changed quite radically over the course of the dual coding versus common coding controversy.

It is also important to distinguish the dual/common coding debate from the better known (at least to philosophers), and ongoing, analog versus propositional debate about imagery, which arose a few years later, in the early 1970s (and will be covered in section 4.4). Although these two controversies did become very entangled, the main points at issue are quite different, and an explicit awareness of this may help us avoid some of the confusions that arose from the entanglement. The analog/propositional debate concerns the nature of imagery itself (to put it very crudely, the analog side thinks mental images are inner pictures, and the propositional side think they are inner descriptions), whereas the dual/common coding debate concerns the functional role played by imagery in the cognitive processes of memory and thought. It may be true that that, amongst cognitive scientists, Dual Coding Theory is most often associated with an analog conception of imagery, and common coding is associated with the propositional conception. However, there seems to be no reason to think that it is incoherent to combine a propositional view of imagery with a form of Dual Coding Theory, as Baylor (1972) and Kieras (1978) have proposed. Also, although perhaps no contemporary theorists combine an analog theory of imagery with a common coding theory of memory, it is arguable that this view was implicit in many of the cognitive theories of former times. For instance, Empiricist philosophers such as Berkeley and Hume (and, very arguably, Aristotle) thought of images as being picture-like (analog), and held that all memories (indeed, all mental contents, all ideas) are images of some sort (common code).

When Paivio initially developed Dual Coding Theory in the 1960s, however, psychological thinking was still dominated by neo-Behaviorism, and the prevailing view was that human memory (where it goes beyond the operant or classical conditioning also seen in animals) depends entirely on words, on inner or subvocal speech in one's native tongue: the common code was taken to be a verbal code. This, however, was soon to change. In the very same era that Paivio's ideas were becoming influential, rapid developments in psycholinguistics, the influence of Artificial Intelligence research, and the rise of the computational information processing approach to cognition were profoundly affecting psychological conceptions of mental representation. Although mainstream memory researchers still mostly focused on memory for verbal material, many came to think of this as encoded in the mind in an abstract format analogous to the “deep structures” of Chomskian linguistic theory or the nested data structures of programming languages such as LISP (Collins & Quillian, 1969; Anderson & Bower, 1973). Thus, by the mid 1970s a very different form of common coding theory had become prevalent. Computationally oriented psychologists began to think of memories as being stored as what they called “propositional representations”, or just “propositions”.

It is important to be aware that this use of the word ‘proposition’ differs, subtly but significantly, from the well established philosophical use of the term, whereby propositions are distinguished sharply from representations (mental or otherwise), and are considered instead to be the underlying, entirely abstract meanings (truths or falsehoods) that representations (such as sentences) may express (Gale, 1967). Thus, following the very influential work of Fodor (1975), philosophers expounding computational theories of mind have usually preferred to talk about “sentential” rather than “propositional” representations, with the understanding that the relevant sentences are to be thought of as expressed not in English (or any other natural, spoken, language) but in mentalese, a computational “language of thought” that is (hypothetically) built into the brain, somewhat as a computer's machine code is built into its CPU. But although it seems to capture their intentions quite well, this philosophical terminology is not widely used by psychologists. In what follows, the philosophical mentalese terminology will generally be favored, but readers should be aware that most of the psychological authors discussed in fact speak of propositional representations.

It is very arguable that the “propositional” common coding theory of memory is more elegant and parsimonious than Dual Coding Theory; certainly it coheres more readily with the broadly computational conception to the mind that remains dominant in cognitive science. Paivio argues, however, that parsimony in terms of representational formats is bought at the cost of a need to posit more types of cognitive processes. He also makes a strong case that Dual Coding Theory has the advantage in its ability to account for the broad range of empirical evidence. The theory has stood up well over the decades both to vigorous conceptual criticism and to many attempted experimental refutations, and Paivio has continued to develop, elaborate, and defend it, periodically reviewing the relevant experimental literature (Paivio, 1971, 1977, 1983a, 1986, 1991, 1995, 2007; Paivio & Begg, 1981; Sadoski & Paivio, 2001 – for less partisan reviews see Morris & Hampson, 1983; Thomas, 1987; Richardson, 1980, 1999).