Favorite Quotes
"For every complex problem, there is a solution that is simple, neat, and wrong." - HL Mencken
====
"I hear and I forget. I see and I remember. I do and I understand." - Confucius
====
"The good physician treats the disease; the great physician treats the patient who has the disease" - Sir William Osler
====
" The best test of a person's character is how he or she treats those with less power." - Bob Sutton
====
"Those are my principles, and if you don't like them - well, I have others." - Groucho Marx
====
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
====
"It is hard enough to remember my opinions, without also remembering my reasons for them" - Friedrich Nietzsche
====
"Anyone can make the simple complicated. Creativity is making the complicated simple." - Charles Mingus
====
"Not everything that can be counted counts, and not everything that counts can be counted." - Albert Einstein
====
"A foolish consistency is the hobgoblin of little minds, adored by little statesman and philosophers and divines. With consistency a great soul has simply nothing to do." - Ralph Waldo Emerson
====
"This ain't no party, this ain't no disco, this ain't no fooling around." - Talking Heads, Life During Wartime
====
"What is hateful to you, do not do to your neighbour. This is the whole Torah; all the rest is commentary. Go and learn it." - Hillel, Talmud, Shabbath 31a
====
"You will never understand bureaucracies until you understand that for bureaucrats procedure is everything and outcomes are nothing." - Thomas Sowell
====
"An idealist is one who, on noticing that a rose smells better than a cabbage, concludes that it will also make better soup." - HL Mencken
====
"If you only have a hammer, you tend to see every problem as a nail." - Abraham Maslow
====
"A great teacher is one who realizes that he himself is also a student and whose goal is not to dictate the answers, but to stimulate his students creativity enough so that they go out and find the answers themselves." - Herbie Hancock
====
"There are no facts, only interpretations." - Nietzsche
====
"An education isn't how much you have committed to memory, or even how much you know. It's being able to differentiate between what you do know and what you don't." - Anatole France
====
"In character, in manner, in style, in all things, the supreme excellence is simplicity." - Henry Wadsworth Longfellow ====
Nov - 15
Dec - 19
Jan - 21
Feb - 8
Mar -
Does peer review help medicine?
3 Feb2010
As readers know I both write article that have appeared in medical journals and I review for many journals. I have twice served as a deputy editor of journals during my career.
Yesterday I received an email about an article I reviewed. The journal sent me all 5 reviews of the article, including mine. They had decided to reject the article. I felt bad, because in my opinion, some of the reviewers had missed the point of the article and the strength of this article.
So last night and this morning I have pondered the problem of peer review. I know that most academics worship at the altar of peer review. I do not. I have seen the problems of peer review in evaluating abstracts, articles and grants.
Too many reviewers become flaw detectives. They focus on the many details (the trees) and fail to appreciate the big picture (the forest). Too often reviewers show their skills at finding flaws or slight weaknesses rather than trying to understand the implications of the paper (or abstract or grant).
I believe that I have done this myself. More recently, I have tried to avoid this tendency, but I am certain that often the authors read my reviews and appropriately curse me. I know that I have cursed those who have given me flawed reviews. Who reviews the reviewers? Why do they have such power.
Now this is not just a philosophical point. Why do we do peer review? I believe the goal of peer review is to strengthen the science we read. Implicitly this view assumes that each article makes a statement that we should absorb.
But science, at its best, should be a conversation. We advance hypotheses based on data and should accept those who challenge our hypotheses. We should make room for those challenges so that we can continue to refine our understanding of medical care and medical science.
We should transform medical journals from speeches to conversations. We should learn to better understand the rationale for an article rather than just criticizing the details.
This transformation has challenges. I am not proposing a clear answer to the problem, but I am calling for others to consider these comments seriously. We at least should have more commentaries on articles and more debate over controversial issues.
Related posts:
- Who watches the watchers? – more on peer review
- More criticism on peer review
- Does peer review work?
- Can expert panels improve health care?
- Criticizing the NEJM
Related posts brought to you by Yet Another Related Posts Plugin.
- In: Medical Rants
5 Responses to Does peer review help medicine?
pb
February 3rd, 2010 at 1:31 pm
I think I'd agree with you more if we didn't have blogs. To me, peer-reviewed journals serve an important role–they provide filtered information. Sometimes the filter is too stringent, sometimes to lax, but usually it ensures good-quality or better studies that are of some significance.
Likewise, blogs serve an important role. They provide unfiltered information–they allow the conversation you refer to. I learn a ton from blogs, and get a lot of enjoyment out of reading them. But I have to do my own filtering.
Peer-reviewed journals are like learning at Grand Rounds. Blogs are like learning from your smart friends over pizza. It's good to have both.
ken covinsky
February 3rd, 2010 at 3:01 pm
Bob—thanks for this really thoughtful discussion. We should subject peer review to rigorous evaluation, just like other processes in medicine.
I am reasonably convinced peer review results in more benefit than harm. I agree totally with the concerns you raised, but I am not sure the situation is quite that bleak.
It is interesting that peer review serves such a vital function in medicine, yet we receive no training in how to do it well, and get little feedback in how we could do it better. LIke you, I have found that I have to be careful about watching my tendency to become to detail oriented, and focus more on the big picture. Warts and all, does this work break new ground, and inform our thinking about this issue?
I have learned that longer reviews are not better reviews. I have also tried to learn to not try to get the authors to write the paper the way I would, and not comment on every issue that comes to mind. It helps when I remember to ask myself, "would my suggestion make the paper substantially better?" If the answer is a just a little better, perhaps I should defer to the authors who have thought about their paper more than me.
On the other hand, at a minimum, peer review serves several very important cultural purposes. I do think there is considerable value when reading a journal in knowing that everything had to pass through at least some quality filter in order to get in. I know I write more carefully because I know my work will have to pass the muster of peer review. I can think of multiple occasions when peer reviewers definitely changed the direction of my work in very positive ways. On the other hand, there are times when I thought the suggestions for change were not desirable, but did them anyways—feeling that the issue was not a matter of principle, and the academic part of me wanted to see the manuscript published.
At least in terms of manuscripts, the downside of peer review may not be that great. If a good manuscript is rejected by one journal, it will almost certainly get into another journal. In terms of grants, the downsides of ineffective peer review is more significant.
I agree with PB that blogs and journals have different function. Blogs are where you can say what you want uncensored—and the comments can be an effective after the fact peer review. Journals should have a peer review filter. But we really need to take the issues you are raising seriously and think about how we can improve the process of peer review and the skill of reviewers.
Jan Krouwer
February 3rd, 2010 at 3:39 pm
I was going to comment, but my comment would have been virtually identical to that of Pb.
Ming Jack Po
February 3rd, 2010 at 7:14 pm
I agree that peer review is a flawed system, but not sure there is a better one at the moment. Very often, there's more politics involved in peer review than actual merit.
Tony Zelinko
February 4th, 2010 at 8:36 am
BoB,
Interesting topic,
In my opinion peer review needs to be understood in it’s context. It is just a review of a piece of work or effort. So often Ego’s get in the way of great works or science and the person submitting the work will take it personally and shut down their creative drive and input. I’ve been on both sides of the coin as a former researcher at Dow. One example I remember is when I used to interpret Japanese patents for my work in the Halogens Research Lab. You see at that time the Japanese would take a specific compounds and patent them individually.This would allow for numerous patents and if you just looked at shear volume, you would conclude they were brilliant and innovative. Ah let’s look deeper, You see here in the US our patent system is different. The same compounds that the Japanese would receive a patent individually for would be looked at in US and classified under specific variations and subsets and specific substituted groups and the amount of patents would be reduced remarkably. I was amazed how the Japanese would just change one halogen substitution and receive a patent. While a US researcher would have to file a generic classification. Now I ask you who has the most brilliant inventors? The Japanese for all their Patents or the US researcher for arranging them in generic classifications. You don’t have to answer, I’ll do it for you, with a twist.
The answer is the researcher who has the best compound to be effective in it’s expected use.
So In a nutshell peer reviews are only as good as the understanding and knowledge of the reviewer. Often times the reviewers lack in both of them.
Good luck with you Blog, and don’t take reviews personally.
Check out mine at my site http://www.bontemedical.com
or go directly to blog http://www.bontemedical.com/blog
Regards,
Tony Zelinko