The debate was conducted on many fronts simultaneously and involved many innovative experiments. To cite just one example, Kosslyn and his collaborator Steven Schwartz responded to Pylyshyn's fears that homuncular sensorium implied a logical paradox by creating a computer simulation that incorporated internal representations of imagery from which propositional content was, in turn, derived. Doing so eliminated Pylyshyn's in-principle objections to the infinitely regressive nature of homunculi. Those interested in more detail and other experimnets are encouraged to consult either Michael Tye's The Imagery Debate or Howard Gardner's The Mind's New Science. Instead, I want to approach the debate with some imagery of my own.
In order to do so, I consulted the bibliography of a major, late-career book on the debate by both Pylyshyn (Seeing and Visualizin: It's Not What You Think, 2001) and Kosslyn (Image and Brain: The Resolution of the Imagery Debate). I then noted every example of either author citing a paper of which he was himself the lead author. Because Pylyshyn is somewhat less prolific than Kosslyn (a function of his running a lab at Harvard), and also because Kosslyn, by mid-career was almost always lead author, whereas Pylyshyn often granted that honor to his frequent co-author and active image debate participant Jerry Fodor, I include Fodor-Pylyshyn collaborations as well. I then dumped the each set of papers into a single .pdf which I in turn fed into IBM's free text visualizing software Many Eyes. I have included links to bibliographies for each author that I compiled in the comments feed.
I first created a wordle, which creates a visual representation of the most common words in a body of texts. Doing so produced the following image for Pylyshyn's work:
Compare that with the wordle of Kosslyn's work:
The first thing one notices is that Kosslyn uses "image," "images," and "imagery" considerably more often than Pylyshyn, but this is perhaps to be expected. Pylyshyn wanting to prove that mental imagery doesn't exist, so one can hardly expect him to use the term as much as Kosslyn, who is defending its existence. The more interesting aberration is the prominent position that "hemisphere" has in Kosslyn's wordle, whereas it does not appear in Pylyshyn's at all. Closer inspection (using Many Eyes' Word Tree Generator) revealed that Kosslyn used the term 521 times, or once every 107 words (an absurdly high rate). By contrast, Pylyshyn uses the word exactly once in the 146,810 words of his articles. "Come, Watson!" I thought. "The game is afoot."So what's so important about hemispheres for Kosslyn, and why is Pylyshyn ignoring them? It turns out that much of the detail of Kosslyn's work that focuses on how the brain structures the relationship between imagery and propositional content involves identifying processing that occurs primarily in the left hemisphere (where more propositional processing happens) or in the right (where more imagery processing occurs). For Pylyshyn, the distinction is moot because images are ultimately reducible to computation anyway. For him, Kosslyn's distinction merely shows where the epiphenomenon is generated. This distinction arose, I think, because Pylyshyn's belief (following Chomsky and Fodor) that the mind need necessarily be computational, led him to be less interested in empirical evidence about the brain itself.
Interestingly, Kosslyn credits Pylyshyn's critiques with urging him and his collaborators to refine their understanding of how the relationship between these subsystems were structured. Ventriloquizing Pylyshyn, Kosslyn pondered "Does the claim that image representations are depictive imply various paradoxes and incoherencies?" He responds that
Addressing this set of issues was no mean task. My strategy was to take advantage of the abundant evidence that imagery piggybacked upon a a theory of high-level visual perception. This approach allowed my colleagues and I [sic] to develop theories of specific processing subsystems, and helped us to hypothesize where these subsystems are implemented in the brain. (Image and Brain 406)Kosslyn, it seems, attributes much of his own successto having been prodded by the Pylyshyn the gadfly philosopher (Kosslyn, btw, is currently a dean at Harvard where he also runs an eponymous research lab and holds a chair dedicated to William James; the prodding seems to have worked quite well). More importantly, though, his research platform has proved more influential than Pylyshyn's, though the latter author remains highly respected as well, having recently won France's prestigious Jean Nicod Prize.
Evidence of this may perhaps be seen in each author's continuing relationship with the iamagery debate itself. Kosslyn has, since the early 90s, stopped defending his approach and concentrated on elaborating its details, whereas Pylyshyn continued to make the case well into the 00s. Perhaps the most telling sign of how the debate was conducted comes again from Many Eyes. Returning to the Word Trees, I checked to see in what contexts each author mentioned the other. Here are Kosslyn's references to Pylyshyn:
You'll notice that of Kosslyn's 60 uses of the Pylyshyn's name, he most commonly references his antagonist in a bibliographic setting, as indicated by the commonness of his name being followed by a common and his initials, but that the lower part of the tree is full of words indicating that Kossylyn is rehearsing one of Pylsyhyn's arguments (e.g. "argues that," "wants," "claims that," "suggests"). Now consider Pylyshyn's references to Kosslyn:
Pylyshyn mentions Kosslyn 138 times, almost twice as often as the other way around, but almost exclusively in a bibliographic context. One possible reason for this is that Kosslyn is, indeed, a more prolific author (again, a function of his running a research lab). This does not, however, explain the absence of contextual references. My suggestion is that this absence may be indicative of Pylyshyn's unwillingess to adapt his model to the criticism of his interlocutors. By excluding them from the explicit discussion, he does not need to frame the debate as a debate, but merely as revealed knowledge. He can occupy a set conceptual ground without appearing entrenched.
Intriguingly, Kosslyn, in recent years, has focused on digital visual communication. Though he has not turned to text visualization software like Many Eyes, he has analyzed strategies for presenting information in different graphical formats, most notably PowerPoint. The move toward engineering and toward applied uses for theory developed during the imagery debate also indicates that the underlying field has stabilized sufficiently to serve as a foundation for new structures.
At present, the debate has moved away from treatments of pure imagery and toward inquiries into more holistic models of perception. This is partly a function of improved research methods, such as head-mounted eye-tracking equipment and improved fMRI technology, partly a function of theoretical refinements, such as attempts by connectionist theorists to understand how the brain's massively parallel micro-structure influences its operation or of dynamicists, such as Tim van Gelder, to understand how the fact of mental computations occurring in real-time must constrain theoretical models. At the same time, Artificial Life roboticists such as Rodney Brooks were constructing robots that performed perceptual tasks with insect-level competency and no internal representations and no central processing whatsoever, offering a critique both to Kosslyn's interest in mental imagery and also to Pylyshyn's insistence in a computational model of mind. Meanwhile, an increasing willingness on the part of psychologists to pay attention to first-hand accounts of phenomenological experience has led to groundbreaking research on subjects ranging from synesthesia (as in the work of Richard Cytowic) to accounts of agency (as in the work of Evan Thompson). This drive toward more holistic descriptions of perceptions is seen also in the work of Alva Noë, who argues that perception, and indeed cognition more generally, must be understood as a fully embodied process; an argument extended to include extra-bodily technologies by philosophers like Andy Clark.
Our understanding of how vision functions is by no means finalized, and by reflecting back on the imagery debate, and the the terms by which it was conducted, we may be able to make more fruitful the debates that currently occupy researchers. The rivalry between Pylyshyn and Kosslyn was one of the most productive in the history of cognitive science, leading to voluminous new research and considerably refined theoretical understanding. Part of the reason for this was the very fact of its being framed as a debate, and part was the different disciplinary approaches that each party brought to the table. Though Kosslyn's position may have proved more influential eventually, it likely could not have done so without Pylyshyn's ongoing critique. In the end, it demonstrates a rivalry can, when conducted well, prove as successful as a collaboration.
