Orac on guidelines

Orac (respectful insolence) has a post which provides another perspective on the guidelines issue which I have focused on this past week – How “evidence-based” is evidence-based medicine?

Hearteningly, it was found that two-thirds of the cardiovascular risk management therapeutic guidelines were based on evidence from RCTs. Less hearteningly, it was estimated that only one-half of these RCT-based guidelines were of “high quality.” What this means is not that the studies used to support these guidelines were not of high quality. Rather, the reason that half of the studies were downgraded from “high” quality when analyzed in the context of the recommendations is because of applicability. The most frequent reason was that an RCT designed to answer a particular question was being generalized to justify the recommendation it supports in a different clinical scenario. Alternatively, results of studies that were carried out in very defined populations were being used to support recommendations in a more general population. In other words, although high quality RCTs can be the basis for several recommendations, the evidence from a single RCT will not support all of the recommendations derived from it equally well, and sometimes developers of guidelines are forced to extrapolate beyond what the RCTs say simply because there is no better evidence available.

The bottom line is that, in this one area at least, if you believe this study, only around 1/3 of the recommendations in a set of consensus guidelines about how to manage cardiovascular risk in three different conditions are based on “high quality” RCT evidence. The study does have significant flaws, such as a small sample size of guidelines examined and only looked at therapeutic interventions, but it’s probably not all that far from the truth, at least as far as it is able to go. I’m surprised that I haven’t yet seen this study trumpeted on websites like NewsTarget or Whale.to as “evidence” that “evidence-based” medicine is not really evidence based, the implication being that so-called “alternative” medicine does just as well.

I am working to explicate my principles for reinventing the guidelines movement. These principles should consider complexity of illness as well as a priorities for which issues require guidelines.

Am I ranting unnecessarily? I think not – read this article from the BBC today – Expensive antibiotics ‘over-used’

The British Pharmaceutical Conference heard one in eight prescriptions in one area were for more modern drugs, often breaching guidelines.

Experts say these drugs should be held back as a “last line of defence” – to cut the risk of bacterial resistance.

Doctors’ leaders said there might be good reasons for GPs’ choices.

I love that phrase – breaching guidelines. This news report based on “experts” demonstrates the semantic drift that the guideline concept has undergone. Experts view guidelines as rules. They are not, and should not be rules.

I am not quite finished with ranting on guidelines. This issue is so important that we need a long explication.