A Problem with Peer Review

"A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather its opponents eventually die, and a new generation grows up that is familiar with it." Max Planck

Nov. 10th's Wall Street Journal had an article on "An Outcast Among Peers Gains Traction on Alzheimer's Cure". A paraphrasing of key comments from the article are:

• Science is politics, and the politics of xyz is winning
• non xyz researchers found it hard to get money, or papers published
• American and British venture capitalists wanted to invest in xyz projects, not zyx
• and noting the comment of a competing researcher:
  if our xyz medical trial results don't fit the hypothesis, we will keep going until they do

We cannot afford for science to become politics, either dominated by public politicians or by peers whose reputations or funding depend on competing approaches. Innovation, technology and science are moving too quickly to wait for the older generation to die off.  I do not know if the xyz, zyx, or mnop approach to curing Alzheimer's will yield the most effective results, but I do know I want all reasonable paths pursued, with a strong outcome oriented approach.

Unfortunately, our research model carries much of the same baggage.  Peer review means seeking input from experts in the field -- often folks who are competing for the same research dollars and whose reputation is tied to years of investment in alternative approaches. No papers published => no funding => no results =>  no start-up (upstart?) company to produce the better mousetrap.

There is a potential path forward here. The web provides a means for publication and commentary -- unfortunately with little quality control (the real purpose of peer-review) to help separate the wheat from the chaff. Professional societies could facilitate the creation of online communities to provide a more open dialog about key topics, and ideally allow "a thousand blossoms to bloom".  This is risky, since it really is simply a "larger" peer review community in some ways, and there is a tendency to place greater credibility on the pronouncements of the venerable icons than may be warranted. However, by providing transparency and open involvement it will be possible for birds-of-a-feather to spin out related communities where they can pursue alternatives.  In such a model we should expect a high failure rate -- however, as many high growth companies know, it is not failing that is a problem -- you want to fail quickly, and at fairly low cost so you can move on to a future success.

There already are online communities of interested persons in many areas.  The advantage of the professional society engagement is two fold: it provides a prima facia pool of qualified experts, and a perceived credibility for the dialog.  No doubt colleagues of mine will argue that this credibility will be lost if the ratio of  'failures' increases, and there is a risk there as well. However, given some high visibility errors from the old guard, it is clear that credibility can be lost in any case.  A few examples may help to clarify the problems we face:

• 100 Scientists Against Einstein (A politically inspired effort in part)
  (Einstein's famous rebuttal -- "If I were wrong it would only take one.")
  [Would Einstein have published his famous 1905 papers if he had been in academia vs the patent office?]
• Lord Kelvins rejection of Darwin's implied age of the earth (it would have cooled off if it were more than 40 million years old)
  and also his skepticism about flight by heavier than air devices
• And similar predictions about horseless carriages, locomotives, and no doubt the wheel and fire

The bottom line is that we can no longer wait for the old guard to die,  nor can we let 'politics' within our communities or from the outside block our progress towards effective solutions.