The Place of Peer Review

by BONNIE WHEELER

[we are pleased to bring to you for discussion this pre-publication section of an essay to appear in the Journal of Scholarly Publishing, "What is a Scholarly Journal? The Place of Peer Review" by  Bonnie Wheeler]

Credentialing / Peer Review
Journal publication provides several forms of credentialing. The first is simply the credit that redounds to an author by being published in a particular journal. The second is the “value added” by careful checking and editing of the article prior to publication. Each editor can list articles that might accurately cite the editor as co-author. The third is the crux: peer review. The credentialing provided by peer review is, I think, the bedrock “value added” in scholarly journals, even though we are as diverse in its definitions as we are about all other aspects of journal editing.

Academic editors think about peer review constantly.(1) Because of petitions and protests, I was alerted to the fact that several science journals still practice single-anonymous review (the reviewer remains unknown but the names of the authors are revealed). I suspect, but have no reliable empirical proof, that most of us in the humanities are accustomed to requiring standard double-anonymous peer reviews even though we are all aware, as bitter colleagues note, with Foucault in the foreground, that peer review is often viewed as “a form of censorship,” an exclusionary, disciplining mode that functions to inhibit or delay new work. I am told that the German system requires a fully open review process in which all parties are revealed to each other. There are strong reasons and feelings on all sides of these arguments.(2)

As an editor, I always feel guilty about requesting double-anonymous peer reviews, knowing that the reviewer will never be fully credited either in print or in profit for a penetrating, helpful peer review. Perhaps I am once again too naïve: do scholarly journals increasingly pay for peer review, or does peer review count in salary negotiations? (3) Probably not. I’ve recently received several complaints (one recently in a powerfully argued editorial from the Journal of Hydrology) over the past years about the increasing unwillingness of specialists to provide peer review precisely because it is “unrewarded activity.” The generations of junior faculty we “protected” from committee work and taught to concentrate on their own rewarded activity have now moved into senior positions. Though many of these colleagues have benefitted mightily from fine peer review, they aren’t necessarily acculturated to accepting reciprocal responsibility. We made ’em this way, we socialized them this way, but now we need them to adopt a different professional training and sociology and “ante up” if peer review is to survive in its current form. Some even suggest a “point system” by which reviewers would earn points they could redeem when they next request peer review for their own work.

Journals that our colleagues recognize as most prestigious will still get their attention (flattery has its uses), and many of you, my fellow editors, are skeptical that there is a problem at all. But on the evidence that I see, an occasional irritation is quickly becoming a dominant force. The pressures increase as the number of journals and thus journal submissions increase. In my experience, most schools offer some stipend when they approach you for tenure and promotion reviews; most presses reward you for manuscript reviews. But for peer review of essay submissions in the humanities, we depend upon the generosity of others.

We depend upon a commitment to our profession that transcends private gain. We used to call it collegiality. Collegial peer review, in all its manifestations, depends upon a fading notion of noblesse oblige. Freely produced peer review may be one of the last “class” vestiges we retain in our profession, an appendix remaining from the positive side of the otherwise punitive (to outsiders) Old Boys’ Network. Yet, in revised formats, it now might provide a transparent activity that reflects one’s obligation to aid the development of vivid intellectual work in one’s field.

How can we remake our systems to interest our young in participating in the credentialing process? In this issue of JSP, James J. O’Donnell suggests that journals of academic societies change their mission:

The traditional publishing peer review ... is seriously flawed and depends too heavily on the willingness of scholars to participate. What vehicles can be found for bringing together scholars around a common purpose and exposing what they do to evaluation of its own? ... Could we imagine a [learned] society that suspended publication of a journal per se but focused on a panel of reviewers who evaluated and recognized the best articles published by members wherever they should appear? A society's web page that contained links to twenty of the best articles published this year by its members might well do more for the discipline than one that spent proportionally more effort on soliciting and editing twenty articles of its own.”(4)

This is an attractive but exclusionary tactic: the notion of “twenty best” blockades the intellectual opulence and openness that our generation of academics has worked hard to achieve.

NIH grant directives insist upon and provide funding for peer review in the sciences. The formal assessment of proposed research begins, according to the NIH, “at a sufficiently early stage to influence the course of that research, the nature of its outputs, and ultimately even whether it takes place at all (or is made available to a wider audience).” In humanities publications, peer review typically takes place at a medial point, when an author or authors submit to a journal an essay that has been shaped to the point that it is considered “ready for publication.” Kathleen Fitzpatrick, whose rich intervention in this question I mentioned earlier, uses an e-publication format that embodies the desire for credentialing that she advocates—a kind of 1960s intellectual e-commune in which everyone participates in the work of production and of evaluation. In her complex analysis of peer review, especially on-line credentialing, she models a new best practice: open peer review, in which scholars post the penultimate version of their work to a website that invites responses. Those responses are open in every sense: the reviewers are known (and thus can be credited), the author can respond, and the final “product” in hard or e-copy acknowledges all participants in the process. She argues that:

the time has come for us to consider whether, really, we might all be better served by separating the question of credentialing from the publishing process, by allowing everything through the gate, and by designing a post- publication peer review process that focuses on how a scholarly text should be received rather than whether it should be out there in the first place. What if peer review ... became peer-to-peer review?

Fitzpatrick’s process may become a new norm, though it is notable that her work, posted now for many months, has not received heavy e-commentary. So far, like our CELJ blog,few are answering her call for comments, contradictions, and suggestions. Her book is slated for hardcopy publication, after which it will presumably receive the prominence it deserves in book reviews.

Whether peer review will continue to fulfill its highest goal of aiding authors to achieve the originality, clarity, and authority they seek depends upon what individual and collective roles we play as editors and publishers. How much of this will remain under editorial control is an open question. But among all elements of scholarly editing, peer review is the most likely to persist in some form. At its best, this form of credentialing extends our mission as teachers to authors who are our most intensely interested audiences. Most fearfully, however, credentialing suits larger political mandates for evaluation and control.

Bonnie Wheeler, Director of the Medieval Studies Program at Southern Methodist University, is executive editor of Arthuriana and past president of CELJ, for whom she collected and edited the essays in this issue of JSP.
--------
(1) For thoughtful studies of peer review, see David Shatz, (Lanham, Md: Rowman & Littlefield, 2004) and Kathleen Fitzpatrick, Planned Obsolescence: Publishing, Technology, and the Future of the Academy, on-line pre-print.

(2) Thatcher comments: “I would hazard the generalization that peer review of monographs carried out by university presses generally fits the single-anonymous review model, though reviewers are always given the option of revealing their identities to the authors.”

(3) The book world, in general, operates differently, and pays at least a nominal amount to reviewers, though I’ve been watching recently as some mega-corporations have begun to compensate journal reviewers with come-ons worthy of carnival hawkers: see Literature Compass.