Tuesday, June 2, 2009

The Imagery Debate, Pt. 2 - Kosslyn v. Pylyshyn

Following Pylyshyn's paper, many psychologists came to the defense of mental imagery, most notably Stephen M. Kosslyn. Over the following three decades, Kosslyn and Pylyshyn (as well as allies on both sides) would conduct a debate over the relationship between propositional and imagistic content in mental processes. Though both sides presently declare themselves victors (of course), the reality of the matter is somewhat hazier, but also more interesting and more fruitful, than a single set of laurels could indicate.

The debate was conducted on many fronts simultaneously and involved many innovative experiments. To cite just one example, Kosslyn and his collaborator Steven Schwartz responded to Pylyshyn's fears that homuncular sensorium implied a logical paradox by creating a computer simulation that incorporated internal representations of imagery from which propositional content was, in turn, derived. Doing so eliminated Pylyshyn's in-principle objections to the infinitely regressive nature of homunculi. Those interested in more detail and other experimnets are encouraged to consult either Michael Tye's The Imagery Debate or Howard Gardner's The Mind's New Science. Instead, I want to approach the debate with some imagery of my own.

In order to do so, I consulted the bibliography of a major, late-career book on the debate by both Pylyshyn (Seeing and Visualizin: It's Not What You Think, 2001) and Kosslyn (Image and Brain: The Resolution of the Imagery Debate). I then noted every example of either author citing a paper of which he was himself the lead author. Because Pylyshyn is somewhat less prolific than Kosslyn (a function of his running a lab at Harvard), and also because Kosslyn, by mid-career was almost always lead author, whereas Pylyshyn often granted that honor to his frequent co-author and active image debate participant Jerry Fodor, I include Fodor-Pylyshyn collaborations as well. I then dumped the each set of papers into a single .pdf which I in turn fed into IBM's free text visualizing software Many Eyes. I have included links to bibliographies for each author that I compiled in the comments feed.

I first created a wordle, which creates a visual representation of the most common words in a body of texts. Doing so produced the following image for Pylyshyn's work:

Compare that with the wordle of Kosslyn's work:
The first thing one notices is that Kosslyn uses "image," "images," and "imagery" considerably more often than Pylyshyn, but this is perhaps to be expected. Pylyshyn wanting to prove that mental imagery doesn't exist, so one can hardly expect him to use the term as much as Kosslyn, who is defending its existence. The more interesting aberration is the prominent position that "hemisphere" has in Kosslyn's wordle, whereas it does not appear in Pylyshyn's at all. Closer inspection (using Many Eyes' Word Tree Generator) revealed that Kosslyn used the term 521 times, or once every 107 words (an absurdly high rate). By contrast, Pylyshyn uses the word exactly once in the 146,810 words of his articles. "Come, Watson!" I thought. "The game is afoot."

So what's so important about hemispheres for Kosslyn, and why is Pylyshyn ignoring them? It turns out that much of the detail of Kosslyn's work that focuses on how the brain structures the relationship between imagery and propositional content involves identifying processing that occurs primarily in the left hemisphere (where more propositional processing happens) or in the right (where more imagery processing occurs). For Pylyshyn, the distinction is moot because images are ultimately reducible to computation anyway. For him, Kosslyn's distinction merely shows where the epiphenomenon is generated. This distinction arose, I think, because Pylyshyn's belief (following Chomsky and Fodor) that the mind need necessarily be computational, led him to be less interested in empirical evidence about the brain itself.

Interestingly, Kosslyn credits Pylyshyn's critiques with urging him and his collaborators to refine their understanding of how the relationship between these subsystems were structured. Ventriloquizing Pylyshyn, Kosslyn pondered "Does the claim that image representations are depictive imply various paradoxes and incoherencies?" He responds that
Addressing this set of issues was no mean task. My strategy was to take advantage of the abundant evidence that imagery piggybacked upon a a theory of high-level visual perception. This approach allowed my colleagues and I [sic] to develop theories of specific processing subsystems, and helped us to hypothesize where these subsystems are implemented in the brain. (Image and Brain 406)
Kosslyn, it seems, attributes much of his own successto having been prodded by the Pylyshyn the gadfly philosopher (Kosslyn, btw, is currently a dean at Harvard where he also runs an eponymous research lab and holds a chair dedicated to William James; the prodding seems to have worked quite well). More importantly, though, his research platform has proved more influential than Pylyshyn's, though the latter author remains highly respected as well, having recently won France's prestigious Jean Nicod Prize.

Evidence of this may perhaps be seen in each author's continuing relationship with the iamagery debate itself. Kosslyn has, since the early 90s, stopped defending his approach and concentrated on elaborating its details, whereas Pylyshyn continued to make the case well into the 00s. Perhaps the most telling sign of how the debate was conducted comes again from Many Eyes. Returning to the Word Trees, I checked to see in what contexts each author mentioned the other. Here are Kosslyn's references to Pylyshyn:

Bee76008-4fa3-11de-84d0-000255111976 Blog_this_caption

You'll notice that of Kosslyn's 60 uses of the Pylyshyn's name, he most commonly references his antagonist in a bibliographic setting, as indicated by the commonness of his name being followed by a common and his initials, but that the lower part of the tree is full of words indicating that Kossylyn is rehearsing one of Pylsyhyn's arguments (e.g. "argues that," "wants," "claims that," "suggests"). Now consider Pylyshyn's references to Kosslyn:

Cceb8f48-4fa4-11de-be77-000255111976 Blog_this_caption

Pylyshyn mentions Kosslyn 138 times, almost twice as often as the other way around, but almost exclusively in a bibliographic context. One possible reason for this is that Kosslyn is, indeed, a more prolific author (again, a function of his running a research lab). This does not, however, explain the absence of contextual references. My suggestion is that this absence may be indicative of Pylyshyn's unwillingess to adapt his model to the criticism of his interlocutors. By excluding them from the explicit discussion, he does not need to frame the debate as a debate, but merely as revealed knowledge. He can occupy a set conceptual ground without appearing entrenched.

Intriguingly, Kosslyn, in recent years, has focused on digital visual communication. Though he has not turned to text visualization software like Many Eyes, he has analyzed strategies for presenting information in different graphical formats, most notably PowerPoint. The move toward engineering and toward applied uses for theory developed during the imagery debate also indicates that the underlying field has stabilized sufficiently to serve as a foundation for new structures.

At present, the debate has moved away from treatments of pure imagery and toward inquiries into more holistic models of perception. This is partly a function of improved research methods, such as head-mounted eye-tracking equipment and improved fMRI technology, partly a function of theoretical refinements, such as attempts by connectionist theorists to understand how the brain's massively parallel micro-structure influences its operation or of dynamicists, such as Tim van Gelder, to understand how the fact of mental computations occurring in real-time must constrain theoretical models. At the same time, Artificial Life roboticists such as Rodney Brooks were constructing robots that performed perceptual tasks with insect-level competency and no internal representations and no central processing whatsoever, offering a critique both to Kosslyn's interest in mental imagery and also to Pylyshyn's insistence in a computational model of mind. Meanwhile, an increasing willingness on the part of psychologists to pay attention to first-hand accounts of phenomenological experience has led to groundbreaking research on subjects ranging from synesthesia (as in the work of Richard Cytowic) to accounts of agency (as in the work of Evan Thompson). This drive toward more holistic descriptions of perceptions is seen also in the work of Alva Noƫ, who argues that perception, and indeed cognition more generally, must be understood as a fully embodied process; an argument extended to include extra-bodily technologies by philosophers like Andy Clark.

Our understanding of how vision functions is by no means finalized, and by reflecting back on the imagery debate, and the the terms by which it was conducted, we may be able to make more fruitful the debates that currently occupy researchers. The rivalry between Pylyshyn and Kosslyn was one of the most productive in the history of cognitive science, leading to voluminous new research and considerably refined theoretical understanding. Part of the reason for this was the very fact of its being framed as a debate, and part was the different disciplinary approaches that each party brought to the table. Though Kosslyn's position may have proved more influential eventually, it likely could not have done so without Pylyshyn's ongoing critique. In the end, it demonstrates a rivalry can, when conducted well, prove as successful as a collaboration.

Monday, June 1, 2009

The Imagery Debate, Pt. 1 - Background

The history of the theorization of mind can be described as a long list of analogies to different technologies. "Aristotle had a wax tablet theory of memory, . . . Leibniz saw the universe as clockworks, [and] Freud used a hydraulic model of libido flowing through the system [coupled with a] telephone-switchboard model of intelligence" (Rumelhart 205). By the 1950s and 60s, early efforts in the Artificial Intelligence community had borne enough fruit that researchers increasingly began to favor analogies to digital computers for the bases of their models. Doing so produced great strides in the psychological community and jumpstarted what has become known as the "Cognitive Revolution".

One of the most significant developments that early cognitive scientists argued for was the rejection of the conceit held by most working within behavioral psychology (then the most prominent research paradigm operating in the United States) that the mind is a blank slate. In particular, Noam Chomsky's veritable trouncing of B. F. Skinner's Verbal Behavior set the tone for much of the research that was to follow. The existence of his proposed universal "generative grammar," Chomsky argued, required as a corollary a module in the brain that could produce such universal structures. Later researchers -- particularly the philosopher Jerry Fodor -- extended this assertion of modularity to cover the brain as a whole, increasing the analogy to different hardware structures in a computer. In order for their information-processing metaphor to work, researchers claimed, the brain had to run on an underlying mathematical descriptions such as the binary code that forms the basis of computer software. Fodor prominently argued for this position in his influential 1975 monograph, The Language of Thought.

Initially, this research paradigm proved extremely successful, but at the same time, it quickly became clear that certain mental phenomena were difficult to describe in its terms. In particular, the computer researcher and philosopher Zenon W. Pylyshyn argued that recent refinements of cognitive theory required psychologists to dramatically rethink their understanding of mental imagery. In his debate-provoking 1973 article "What the Mind's Eye Tells the Mind's Brain," Pylyshyn claimed that the new research paradigm revealed imagery to be merely epiphenomenal. That is to say, mental imagery has a phenomenological or experienced reality, but it is inessential, a mere side effect of "invisible" subconscious processing to which the mind cannot be made privy. He argued that mental imagery bore roughly the same relationship to the "real" mental process as the images on a computer screen bear to the computations that occur within a computer processor.

He presented two primary arguments for this position. In the first case, he argued, those who believed in the "reality" of mental imagery implicitly reinstated the Cartesian dualism between mind and body. This position was a reaction to then-recent ground-breaking vision research, most notably the incredibly influential 1959 paper (which inspired the title of Pylyshyn's), "What the Frog's Eye Tells the Frog's Brain," in which a team of international researchers led by the MIT psychologist Jerome Lettvin showed that much important vision processing in a frog actually took place along in the frog's eye. Cells on the retina that responded to small, fast moving objects (which Lettvin et al. deemed "bug detectors") triggered the response of attacking the triggering object with the frog's tongue. The fact that the frog's eye and not the frog's brain seemed to do the work of identifying its perceptual object showed that researchers could advance mechanistic descriptions of cognitive systems that could also account for where meaning come from without needing also to posit some sort of non-mechanical "mind" which attributed meaning and made judgments. This research was then replicated (in somewhat less bug-centered terms) in many other animals including humans.

Pylyshyn claimed that those who believed in the reality of mental imagery gave up the ground that Lettvin and his colleagues had gained. He saw mental imagery as a return to dualism, arguing that the mind showed no sign of containing an additional "eye" with which to view the represented image and also reiterating the philosopher's classical critique of dualism: that it implied an infinite regression of nested homunculi. After all, if the brain required an interior structure to view the represented image, then that internal structure presumably required an observing internal structure as well, which in turn required, ad infinitum.

Pylyshyn's other major claim was that the dichotomy then posited by many vision researchers that mental content was either propositional or perceptual was too rigid. Indeed, he claimed, since perceptual content was ultimately reducible to propositional content, the distinction was meaningless. The renowned British opera director, Jonathan Miller, who was originally trained as a neurologist, made a similar (albeit softened) claim some years later:
I recently went to Cambridge to deliver a lecture, and in my apprehension I had rehearsed my arrival in a series of increasingly alarming dreams. About four nights before, I dreamed that I was parking my car in what I knew to be Trinity Lane, although it was from its visible appearance also a narrow side street behind the Santo in Padua. At the end of the lane, I could see the master of Trinity waving hospitably at me, and he was the actor Michael Horden. Now there was no sense in my dream that Horden, the actor, was playing the part, or that he had popped in between the election or anyone else's hopes. As far as I was concerned in the dream, Horden and the master were one and the same, in spite of the fact that I also knew, simultaneously, that the master and Sir Andrew Huxley were identical. The rest of the dream was so humiliating, for which I blame neither Sir Michael nor Sir Andrew, that I shall draw a veil over it. (198)
Miller's point is that his mental image of Michael Horden also contained the propositional content "master," thus blending the two categories. This, it must be stressed, is a softer claim than Pylyshyn's, Miller not being interested in proving the epiphenomenality of images, but it operates along the same vectors.

Pylyshyn's was the first salvo in what would become a proplonged debate over the nature of mental imagery.

[Sources appear in the Comments feed]