How much do Wachter and I disagree about measurement


Category : Medical Rants

This tweet stimulated a blog post – @HC_TheJournal: @Bob_Wachter on "good enough" quality measures: @medrants seems to have different take:”

What does this tweet mean?

Yesterday I commented on a NY Times article about the impact of Medicare's readmission rewards and penalties.  Bob Wachter wrote about the same problem.

Wachter's blog post was much more accepting of the impact of Medicare's implementation of a readmission measure than my post.  As I read his post, I believe we see the world similarly, but he is more lenient about accepting a measure and then correcting the measure.

I will focus on this paragraph:

Does the public’s tolerance for misclassification give measurers – the ABIM, Leapfrog, or Medicare – a free ride on the “Ends-Justify-The-Means” Express? Absolutely not. Measurers need to do their honest best to produce measures with as much scientific integrity as possible, and commit themselves to improving the measures over time. Medicare’s decision to ditch their four-hour door-to-antibiotic time pneumonia measure in the face of evidence of misclassification and unanticipated consequences (antibiotics at triage for everyone with a cough) is a shining example of responding to feedback and new data. In a recent NEJM article, Joynt and Jha recommend a few simple changes, including taking into account patients’ socioeconomic status, that could improve the readmission measure. The trick is to adjust appropriately for such predictors without giving safety net and academic hospitals a pass, since these organizations undoubtedly vary in performance and many have room for improvement.

Here is the contrast.  I believe that the initial enthusiasm for the four-hour rule represents the acceptance of unintended consequences.  Would we accept the FDA ignoring the side-effects of a new drug or a new procedure?

My key point is that measurement changes medical care, just like a new medication or a new procedure.  If measurement changes how we care for patients, then we have a moral responsibility to FIRST DO NO HARM.

Ditching the 4-hour pneumonia rule after we learn that it does harm is not good enough.  Of course Medicare changed – although the first tried a 6-hour rule if my memory does not fail me.  But what harm occurred because of that rule.

What harm has occurred from other measurement proclamations?  Why would it hurt to prospectively investigate a proposed measurement?  

Logic is not good enough.  We once thought it was logical to give antiarrythmic drugs to patients after they had a myocardial infarction.  We once thought that beta blockers were contraindicated in systolic dysfunction.  We consistently follow logic and conventional wisdom, yet find out that we were wrong.  

Why should performance measures receive a pass on evaluation?

Comments (5)

Not to argue performance measures should get a pass, they shouldn't, but isn't the obvious problem no one can anticipate all the consequences of measuring something and what the effects of that measurement will produce? How do you get around that?
Certainly they deserve rigorous scrutiny before implementation, but a realistic appraisal might conclude that since any measurement might produce harm in some way, no measurement should be implemented.
This, of course, begs the question, which Wachter skirts, of whether performance measurements are the way to go in the first place. But that is another day's discussion.   

Most quality measures are biased in several ways with the bias often subtle. For example, is there a financial incentive in the quality measure? The measures will be said to scientifically based but the bias remains notwithstanding. A quality measure could be compared to a new drug. The data to approve the drug is a small percentage of the data that is available after approval. So it is important that there is a mechanism to modify the quality measure based on its effect.

If you study it you will understand the consequences.

Not directly related to this post, but I think related to your blogs general theme:
I recently read a quote by Marc Andressen (co-inventor of the web browser):
"The spread of computers and the Internet will put jobs in two categories, people who tell computers what to do, and people who are told by computers what to do."
I think this is extremely insightful and maybe even true. I think a large portion of your blog boils down to the question of what category physicians will be in.  Currently our profession is moving toward a computerized model but the transition isn't complete. My EMR has some orders flowing in both directions. I tell the pharmacy computer what prescription should be filled. My EMR tells me every single 18+ female patient is due for a pap smear, regardless of other factors. (or more helpfully, when my diabetics are due for labs)
Will the surgeon tell the computer to get the OR ready? Or will a computer tell the surgeon it's appropriate to operate?
Or in a more hyperbolic sense: Do we want physicians to be closer to engineers? Or closer to fast food employees, taking orders from a glowing screen?

Physicians actually practicing medicine can easily anticipate most of the unsavory consequences of performance measures, even before they are implemented.  
But no one asks them what they think.  

Post a comment