TH!NK post

Gaming the Peer Review System: Part 2. Exploiting Loopholes

Published 04th February 2012 - 1 comments - 1103 views -

There is evidence that the authors of a recent paper may have gamed the peer review system to publish a biased climate science paper.

The Review Process: When a paper is submitted to a journal for publication, the editor removes the name of the author and sends the manuscript to several experts in the area, usually three, for review. The editor keeps the names of the reviewers confidential. If an error is found, the reviewer’s comments are returned to the author with suggestions for corrections. It is a good system for ensuring the quality of research publications, but even then papers are sometimes published that contains errors. The reviewers may miss an error, a biased editor may publish the paper in spite of flaws, or authors may exploit loopholes in a journal’s rules to get a paper published. Some journals allow the author to suggest names of reviewers and the editor often picks reviewers from the list. Most scientists submit names of reliable reviewers as it is an embarrassment to have errors found in their paper after publication. However, even if the papers are properly reviewed, the practice can bring accusations of “pal” review. Since reviewer’s names are kept confidential by the editor, it is difficult to know for sure whether that may have happened. However, there is evidence that the authors of a recent paper may have gamed the system by suggesting a set of reviewers that shared their bias. See what you think.

The paper: Last July 25th, Roy Spencer and Danny Braswell authored a paper in the rather specialized technical journal, Remote Sensing, titled “On the Misdiagnosis Of Surface Temperature Feedbacks From Variations In Earth’s Radiant Energy Balance“.  The paper claimed “The sensitivity of the climate system to an imposed radiative imbalance remains the largest source of uncertainty in projections of future anthropogenic climate change. Here we present further evidence that this uncertainty from an observational perspective is largely due to the masking of the radiative feedback signal by internal radiative forcing, probably due to natural cloud variations.”  It seems that only an expert in climatology would know what that means or what its implications were, but in three days a sensationalized version of the paper appeared on internet sites, in major business magazines, and in news articles in major newspapers. Millions of people likely read about the paper.

The Publicity: The renewed public interest in science should have made climate scientists pleased; however, they were not. Beneath the technical language is a claim that the climate sensitivity to CO2 has been misinterpreted by climate scientists because of natural cloud variations. Were it true, it would mean that natural forces, not man, were responsible for much of the observed global warming. That idea had been examined before and found to be inconsistent with the evidence, but the idea is one that some climate skeptics have been promoting. And, they are part of a well-funded pipeline that carries misinformation about climate science to major news outlets before all the facts can be known.

Forbes: One main branch of the misinformation pipeline runs through the Heartland Institute, where James Taylor is listed as a senior fellow. James Taylor once wrote articles for the tobacco industry suggesting that secondhand smoke was not harmful, and he has now turned his talents to denying the ties between rising CO2 levels and global warming. Inexplicably, James Taylor has been hired by Forbes magazine to write on energy and environmental topics. James Taylor picked up on Spencer’s paper and wrote an article for Forbes titled, New NASA Data Blows Gaping Hole In Global Warming Alarmism. Not only was the title inaccurate and misleading, but the article was clearly an opinion article, miscategorized as news.  The editors of Forbes might not have known that Spencer’s “NASA Data” was the same data that climate scientists use to reach a very different conclusion, but perhaps they should have noticed that no reasonable news story would describe climate scientists as “alarmists” 15 times. The business community considers legislation that would reduce our carbon emissions to be anti-business, and business newspapers such as Investors Business Daily, the Wall Street Journal, and Forbes often are biased toward the skeptic’s position. The bias shows up in story selection, opinions miscategorized as news, a disproportionate number of skeptics articles on opinion pages, and  in sensationalized headlines. From Forbes, the article was picked up as a news story by other business magazines, Yahoo! News, MSNBC, and skeptic’s blog sites, which had a field day with the article. It is sad that millions will have read the distorted article, but few will ever read the climate scientist’s rebuttal. The article will soon sink into obscurity,  but it will have accomplished it’s purpose, which was to spread doubt about climate change.

Reproducibility: Publication in a peer-reviewed journal is not the only requirement for a paper to become accepted as part of the science literature. The research must stand up to the scrutiny of other experts in the field and it must be reproducible by other scientists with comparable knowledge and skill. Spencer’s paper reached the news media before climate scientists had a chance to respond, but they soon found a number of obvious errors in the paper. Trenberth and Fasullo summed it up:”The model has no realistic ocean, no El Niño, and no hydrological cycle, and it was tuned to give the result it gave. The bottom line is that there is NO merit whatsoever in this paper.”  Given time, A.E. Dessler analyzed Spencer’s paper in detail and published a rebuttal. The abstract in Geophysical Review Letters reports the key points of his paper:

  • Clouds are not causing climate change;
  • Observations are not in disagreement with models on this point;
  • Previous work on this is flawed;  ( referring specifically to Spencer’s paper in Remote Sensing).

Clearly, Spencer’s paper had serious methodological flaws and was not reproducible. How did the paper get through Remote Sensing’s peer review process? The answer would likely not have been found, except for the publicity.

The Catastrophe: The editor of Remote Sensing, who had been trying to build the reputation of the Journal, considered the publicity a catastrophe. The instructions in Remote Sensing asks authors to suggest five reviewers, and it is possible that Spencer could choose five skeptics.  The editor would not have to pick from those, but apparently in this case he did.  In the next issue of Remote Sensing, the editor, Dr. Wolfgang Wagner, resigned and issued a public apology for this article saying, “With this step I would also like to personally protest against how the authors and like-minded climate skeptics have much exaggerated the paper’s conclusions in public statements.” “The problem is that comparable studies published by other authors have already been refuted in open discussions and to some extent also in the literature, a fact which was ignored by Spencer and Braswell in their paper and, unfortunately, not picked up by the reviewers. “ And he concluded, “But, as the case presents itself now, the editorial team unintentionally selected three reviewers who probably share some climate sceptic notions of the authors.”

© 2012 J.C. Moore

 

Category: Climate Reporting, | Tags:



Comments

J.C. Moore on 04th February 2012:

Roy Spencer’s blog contained his response to Dr. Wagner’s resignation. He was asked in the comments who he requested as reviewers, but his reply evaded the question:
“1) Who reviewed your paper Roy? Did you submit a list of potential reviewers? If so who, and were they amongst the reviewers selected to review your paper?
(Spencer) re Q1: Almost every journal requires a list of suggested reviewers, and except for one reviewer, the identities of the reviewers chosen was unknown to us.”

I’m not sure how Spencer knew the name of one of the reviewers chosen, but surely he would know the names of the 5 he submitted.

http://www.drroyspencer.com/2011/09/editor-in-chief-of-remote-sensing-resigns-from-fallout-over-our-paper/

  • Remember my personal information

    Notify me of follow-up comments?

    --- Let's see if you are human ---

    What is the fourth word of this sentence? Add a questionmark to your answer. (7 character(s) required)