An academic scandal:
The importance of peer review
by Gary Mauser
November 11, 2002
The Bellesiles scandal is attracting media attention in the US, but it has barely shown up on Canadian radar. Michael Bellesiles, an historian at Emory University in Atlanta, won the prestigious Bancroft Prize in 2001, but he has now been forced to resign from his university position, because charges of "fraud" and "intentional deception." Columbia University reportedly is considering withdrawing his Bancroft Prize.
Despite the fact that this scandal concerns American history, it raises an important challenge for Canadian academics. How could allegedly fraudulent research, not just get published, but win prestigious academic prizes? The rise and fall of Bellesiles opens a window on a more general problem, the fragility of peer review. According to professor emeritus David Bordua, at the University of Illinois, the real scandal concerns the willing gullibility of ideological reviewers and academic historians. The Bellesiles case represents a monumental failure of peer review. When his work was first published, reviewers hailed the book apparently without out bothering to evaluate the research. The book even won the Bancroft Prize without anyone on this prestigious panel bothering to check his footnotes thoroughly. Isnt this something academics are supposed to do with run-of-the-mill undergraduate papers?
In 2000, Bellesiles was being fawned upon by academic and popular critics alike for his book Arming America: the Origins of a National Gun Culture. Peter Onuf, the author of Jeffersons Empire, opined that "Michael A. Bellesiles moves to the front rank of American historians with this deeply researched, brilliantly argued, energetically written, and timely book." Robert J. Spitzer, author of The Politics of Gun Control, claimed the book was, "Meticulously, even extravagantly, researched,
" Michael Zuckerman, the author of Peaceable Kingdoms even said, "This is stunning history, brilliantly argued. It throws into a cocked hat our most cherished assumptions about guns and gun culture in early America." Today, these critics are silently eating crow.
What was all the fuss about? In his book, Bellesiles claimed to have examined over 11,000 probate records between 1765 and 1850 and found a surprisingly small number of firearms. He concluded that firearm ownership was less widespread than previously believed, so that, before the 1860s, a widespread "gun culture" didnt exist. More provocatively, he claimed that the onset of the US Civil War, and the rapid growth of the arms industry, gave birth to the distinctive American gun culture. If true, this would undercut the myth of the American "minuteman" who plays an important role in American political theory.
This is heady stuff. To many historians and social critics, this study supported their prejudices against firearms and their distrust of industry. Without checking too closely, many rushed to applaud Bellesiles as their new hero.
Unfortunately for Bellesiles, cracks soon began to appear in his argument. First, legal researchers challenged his sources. When these scholars checked the sources they couldnt confirm his findings. For example, when professor James Lindgren, Professor of Law at Northwestern University, tried to check the records, he found glaring discrepancies. Bellesiles records either werent where he said they would be or, if found, they differed substantially from what Bellesiles said. Other scholars also checked; all found it impossible to get Bellesiles to share his sources.
Worse was in store. When the records were checked further, many of the records Bellesiles claimed existed couldnt be found. When asked, he had various excuses, each more implausible than the ones hed used earlier. All however were variations of the dog ate my homework. Bellesiles told professor Lindgren that hed kept his notes on yellow foolscap and had no records on disk. This is surprising, but then he said his notes had been destroyed when thered been a flood in his office. However, professor emeritus Jerome Sternstein at Brooklyn College wasnt impressed. Particularly troubling, archives where Bellesiles claimed to have taken extensive notes had no record hed ever been there. His goose was cooked when he was confronted with claiming to have studied records that were lost in the great San Francisco earthquake and fire of 1906.
His academic critics have now painstakingly shown that Bellesiles made up or grossly misrepresented much of his data. The academics who nominated him for the Bancroft Prize are no longer supporting him. The American Society of Criminology scheduled a panel to examine the scandal, but no one could be found to defend him. The National Endowment of the Humanities has acted to remove its name from Bellesiles fellowship. After a six-month investigation, Emory University has now placed him on leave pending their decision about the charges of academic dishonesty.
Academics such as professor Bordua argue that the ease with which his book was accepted by academics is even more shocking than Bellesiles' deficient scholarship. How could anyone be awarded the Bancroft Prize without any serious effort being made to corroborate his research? The real scandal is that not that a professor would conduct fraudulent work, but that his peers accepted it. The most plausible explanation is that ideologues believe the end justifies the means.
In Bellesiles' case, historians eagerly allowed themselves to be hoodwinked. The charitable interpretation is that academics, like anyone else, are not as critical of arguments that support their prejudices as they are with those that are less comforting. This is just human nature. But, academics are supposed to be professionals. Every discipline has set up methodological standards to guard against such human failings. In order to be published, academic books and articles must survive rigorous criticism in the peer-review process. If we do not honorably adhere to the highest disciplinary standards, our research loses credibility.
A less charitable interpretation is that academics are more cynical. Historians may have willingly supported Bellesiles outrageous claims, even knowing they were implausible, because doing so furthered their political objectives. Historians tend to consider themselves "progressive," and so they share a dislike of firearms. Thus, they willingly flocked to support Bellesiles in order to promote their ideological agenda.
Some academics defend such cynicism, arguing that there is no such thing as truth. Everything is subjective. Unfortunately, this standard too easily promotes propaganda. Academic prizes are seen as rewards for your friends or fellow travelers. As one European intellectual told me, "The law only exists to be applied to your enemies." I believe such a cynical approach undermines faith in all academic research.
Dont misunderstand me. This problem isnt limited to history. Nor is it limited to liberals. The Bellesiles case illustrates how crucial peer review is to all academics Canadian or American, historians or scientists. If peer review can fail in history, it can fail anywhere. Recently, the Journal of the American Medical Association admitted that there are serious problems with peer review in medical publications. All academics must work to keep the peer review process healthy and act vigorously to root out unprofessional conduct.
There is no happy ending to this story yet. Peer review did eventually uncover the gross distortions, but this took years longer than it should. Emory University asked him to resign after an independent committee found that he was "guilty of unprofessional and misleading work." Unfortunately, the American Historical Association does not appear to be listening. It is too soon to know if the flaws in the peer-review process that failed with Bellesiles have been recognized, let alone fixed.
Gary Mauser is a Full Professor in the Faculty of Business Administration and a Research Fellow in the Institute for Canadian Urban Research Studies. In 2001, he won the Nora and Ted Sterling Award for Controversy.