Comments on: Does peer review help medicine?

By: Tony Zelinko

Tony Zelinko — Thu, 04 Feb 2010 13:36:37 +0000

BoB, Interesting topic, In my opinion peer review needs to be understood in it’s context. It is just a review of a piece of work or effort. So often Ego’s get in the way of great works or science and the person submitting the work will take it personally and shut down their creative drive and input. I’ve been on both sides of the coin as a former researcher at Dow. One example I remember is when I used to interpret Japanese patents for my work in the Halogens Research Lab. You see at that time the Japanese would take a specific compounds and patent them individually.This would allow for numerous patents and if you just looked at shear volume, you would conclude they were brilliant and innovative. Ah let’s look deeper, You see here in the US our patent system is different. The same compounds that the Japanese would receive a patent individually for would be looked at in US and classified under specific variations and subsets and specific substituted groups and the amount of patents would be reduced remarkably. I was amazed how the Japanese would just change one halogen substitution and receive a patent. While a US researcher would have to file a generic classification. Now I ask you who has the most brilliant inventors? The Japanese for all their Patents or the US researcher for arranging them in generic classifications. You don’t have to answer, I’ll do it for you, with a twist. The answer is the researcher who has the best compound to be effective in it’s expected use. So In a nutshell peer reviews are only as good as the understanding and knowledge of the reviewer. Often times the reviewers lack in both of them. Good luck with you Blog, and don’t take reviews personally. Check out mine at my site http://www.bontemedical.com or go directly to blog http://www.bontemedical.com/blog Regards, Tony Zelinko

By: Ming Jack Po

Ming Jack Po — Thu, 04 Feb 2010 00:14:57 +0000

I agree that peer review is a flawed system, but not sure there is a better one at the moment. Very often, there's more politics involved in peer review than actual merit.

By: Jan Krouwer

Jan Krouwer — Wed, 03 Feb 2010 20:39:31 +0000

I was going to comment, but my comment would have been virtually identical to that of Pb.

By: ken covinsky

ken covinsky — Wed, 03 Feb 2010 20:01:13 +0000

Bob—thanks for this really thoughtful discussion. We should subject peer review to rigorous evaluation, just like other processes in medicine.
I am reasonably convinced peer review results in more benefit than harm. I agree totally with the concerns you raised, but I am not sure the situation is quite that bleak.
It is interesting that peer review serves such a vital function in medicine, yet we receive no training in how to do it well, and get little feedback in how we could do it better. LIke you, I have found that I have to be careful about watching my tendency to become to detail oriented, and focus more on the big picture. Warts and all, does this work break new ground, and inform our thinking about this issue?
I have learned that longer reviews are not better reviews. I have also tried to learn to not try to get the authors to write the paper the way I would, and not comment on every issue that comes to mind. It helps when I remember to ask myself, "would my suggestion make the paper substantially better?" If the answer is a just a little better, perhaps I should defer to the authors who have thought about their paper more than me.
On the other hand, at a minimum, peer review serves several very important cultural purposes. I do think there is considerable value when reading a journal in knowing that everything had to pass through at least some quality filter in order to get in. I know I write more carefully because I know my work will have to pass the muster of peer review. I can think of multiple occasions when peer reviewers definitely changed the direction of my work in very positive ways. On the other hand, there are times when I thought the suggestions for change were not desirable, but did them anyways—feeling that the issue was not a matter of principle, and the academic part of me wanted to see the manuscript published.
At least in terms of manuscripts, the downside of peer review may not be that great. If a good manuscript is rejected by one journal, it will almost certainly get into another journal. In terms of grants, the downsides of ineffective peer review is more significant.
I agree with PB that blogs and journals have different function. Blogs are where you can say what you want uncensored—and the comments can be an effective after the fact peer review. Journals should have a peer review filter. But we really need to take the issues you are raising seriously and think about how we can improve the process of peer review and the skill of reviewers.

By: pb

pb — Wed, 03 Feb 2010 18:31:09 +0000

I think I'd agree with you more if we didn't have blogs. To me, peer-reviewed journals serve an important role–they provide filtered information. Sometimes the filter is too stringent, sometimes to lax, but usually it ensures good-quality or better studies that are of some significance.

Likewise, blogs serve an important role. They provide unfiltered information–they allow the conversation you refer to. I learn a ton from blogs, and get a lot of enjoyment out of reading them. But I have to do my own filtering.
Peer-reviewed journals are like learning at Grand Rounds. Blogs are like learning from your smart friends over pizza. It's good to have both.