Quality measurement – a delusion

26 Apr
2009

Everyone wants to insure quality.  Quality is job #1.  Who can be against quality?  We should measure quality, report it, reward it, and publicize it.

But what is quality medical care?  Can we measure it easily?

Many argue that we can measure quality.  My friend and colleague, Bob Wachter, argues for quality measurement – On Quality Measurement, Babies, and Bathwater

As for me, I’ll keep critiquing bad measures and pointing out when new science emerges that changes how we should think about existing measures. But I’ll continue to support thoughtful implementation of transparency programs and experiments using P4P. From where I sit, of all our options to meet the mandates to improve quality and safety, tenaciously clinging to a Marcus Welbian (and demonstrably low quality) status quo or creating tests that can be passed by appearing to be working on improvement seem like two of the worst.

I have often written about the problems of quality measurement.  But given that I always have new readers, I will reiterate my position.

Imagine that you are picking a general internist.  What attributes would you desire?

As I consider my ideal internist, I would expect excellence is several domains:

  1. Bedside (or chair side) manner
  2. Appropriate time for my appointment
  3. Accurate, complete history taking
  4. Ability to perform physical examination
  5. Understands lab test interpretation
  6. Excellent medical knowledge
  7. Diagnostic acumen
  8. An understanding of the natural history of disease
  9. Knows when to consult other physicians
  10. Evidence based treatment of existing conditions

Now I may have not completed the list, but you get the idea.  Quality, if definable, is a multi-dimensional concept.  Yet most quality measurement (more accurately performance measurement) focuses just on dimension #10.  Now it gets even more complex.

As a physician I care for 100 patients this week.  25 of these patients have measurable treatments.  So in a report card, I am measured on only 1/4 of my patients.  Now it gets even more complex.

As a physician, I am often measured according to patient results (intermediate outcomes), such as HgbA1c, BP, LDL cholesterol.  So we have even another variable – the patient.  I am now responsible for the patient’s adherence.  Patient adherence has multiple influences. 

Now, I am not against an attempt to improve my measurable characteristics.  I try to provide the best possible care.  But we should use measurement on a personal level, and only when that measurement has proven to make a difference.

Wachter provides multiple examples of failed performance measures.  Those who favor performance measures explain the failures as errors in development.  Prominent advocates have argued that we do not need evidence if efficacy prior to adopting a performance measure.

This attitude would invoke laughter if it was not so dangerous.  The performance czars believe in measurement, and cannot imagine that it will not work.  I argue that there viewpoint is askew.  These bean counters do not really understand the complexity of medical care.  They live in a one dimensional world, and cannot imagine the other dimensions.

They forget Einsteins’s admonition – Not everything that can be counted counts, and not everything that counts can be counted.  They have not studied Onora O’Neill – In theory again the new culture of accountability and audit makes professionals and institutions more accountable for good performance. This is manifest in the rhetoric of improvement and rising standards, of efficiency gains and best practice, of respect for patients and pupils and employees. But beneath this admirable rhetoric the real focus is on performance indicators chosen for ease of measurement and control rather than because they measure accurately what the quality of performance is.

I will continue to rant against performance measurement except as a self improvement tool.  When we use performance measurement, we distort physician practice.  We provide too much focus on 1 dimension of quality, and thus the other dimensions must suffer. 

This issue will continue to anger me, because the lack of understanding is mind boggling.  I want to provide the highest quality care.  Numbers cannot capture or measure my successes or failures.

 

Related posts:

  1. Measurement – the good and bad
  2. Safety rather than quality
  3. What is quality – a reader asks
  4. Can we measure quality?
  5. Anonymous creates a strawman argument of P4P

Related posts brought to you by Yet Another Related Posts Plugin.

9 Responses to Quality measurement – a delusion

Avatar

brian

April 26th, 2009 at 8:55 pm

fact is: PQRI and quality measures are thinly veiled attempts by Medicare and insurance to have another excuse to either cut payments or NOT pay at all.

it is a joke to think it has anything to do with improving quality of care.

cookbook medicine has never worked and it is a sad day when we treat ALL patients the SAME. based on a protocol.

Avatar

Topics about Laughter | Quality measurement - a delusion

April 26th, 2009 at 9:57 pm

[...] DB’s Medical Rants added an interesting post on Quality measurement – a delusionHere’s a small excerptEveryone wants to insure quality.  Quality is job #1.  Who can be against quality?  We should measure quality, report it, reward it, and publicize it. But what is quality medical care?  Can we measure it easily? Many argue that we can measure quality.  My friend and colleague, Bob Wachter, argues for quality measurement – On Quality Measurement, Babies, and Bathwater .  As for me, I’ll keep critiquing bad measures and pointing out when new science emerges that changes how we should thi [...]

Avatar

country solo doctor

April 27th, 2009 at 7:12 am

As a solo doctor, I tried quality measurements with Medicare on LDL cholesterol, A1C, and BP on diabetics. It became a hassle to report each of these on patients for a massive 2% bonus on office fees, only on the reported patients and only after meeting a certain percentage of reporting. I figued a third of my Medicare patients have diabetes and an average office visit of 4 times a year each. The bonus was going to be around $900 a year, which equates to a 6 day work week and 3-9 Medicare patients/day of less than $2 a day or less than 50 cents a patient. It was not worth my time.
So far, none of the private plans have based my fees on quality, but three of my national plans are tracking quality, such as mammograms, pap smears, LDL cholesterol, A1C, urine microalbumin, etc. I went from 70% average on labs to over 90% average, when I started doing the labs in my office. At least 20% of my patients were not doing the ordered labs. Many of patient refused to do colonoscopies for cancer screening at age 50 or older, as they have $1,000-$5,000 deductibles. Other patients have ob/gyns, yet I get dinged when a mammogram or pap/thin prep is not done. Finally, I can start with a new patietn with an A1C of 12-14, severely uncontrolled diabetes, and get that A1C down within 6 months to a year to 7-8 range, yet that person is still considered uncontrolled diabetes by quality ranking. Ditto if a patient as an LDL cholesterol of 150 with diabetes, high chol, and HTN, and I get it down with meds/diet/ex to 110, that patient still has not med national guidelines. Quality if vague across the plans.
One other thought, a physician who does too many referalls or tests in Medicare Complete gets up to a 10% ding on his/her fees while receiving monthly capitation payments regardless of seeing patients. Capitation encourages physicians not to see the patients by financial incentives. Quality with Medicare complete and most capitated plans is simply being an efficient doctor.

Avatar

James Gaulte

April 27th, 2009 at 8:01 am

Onora O’Neill’s comment that quality indicators are chosen for ease of measurement and control and do not measure accurately what the quality of performance is ranks as a great insight but I think even more fundamental to the issue is Goodhart’s law.This states that when a measurement becomes a target it no longer is a valid measure. This is largely true because people respond to incentives.

Avatar

cory

April 27th, 2009 at 8:10 am

I would amend your point just a little.
Quality, from any scientific management perspective, is quit easy to define.
Quality is conformance to predefined standards.
The problem we have, as your commentators have noted, is not defining quality but defining standards. Many, if not most, if not all the standards currently used are flawed in some way. Our job is to “unflaw” them as it were, admittedly not an easy task.

“Doing the right thing is easy. Knowing what the right thing to do is the hard part.”
– Lyndon Johnson

Avatar

docanon

April 27th, 2009 at 8:26 am

Could you please provide us with a list of the “performance czars”? It doesn’t have to be a complete list. Just name the first few that come to mind.

I wonder if they really hold the beliefs you attribute to them. Maybe you could consider asking. Or maybe baseless accusation is more fun.

Your take on this whole subject smacks a bit of Glenn Beck, and your level of paranoia is deeply concerning. In fact, I feel it necessary to take the specific step of pointing out that nobody is–or has been–threatening you.

Avatar

pcb

April 27th, 2009 at 8:27 am

Docs who work in more affluent areas, on average, will have higher P4P scores.

Docs who mislead patients, exaggerate benefits, and encourage treatments/meds based on simplistic number goals will have higher P4P scores.

Docs who refuse to consider whether the patient in front of them appropriately fits the guidelines being measured will have higher P4P scores.

Docs who avoid shared decision making (what if the patient’s informed decision goes against what’s good for my scores?) will have higher scores.

Docs who try to avoid or refer out difficult patients or those with difficult disease will have higher scores.

So if you want to encourage docs to flock to areas with the healthiest patients, avoid those who need the most help and are often the hardest to treat, mislead patients to achieve simplistic number goals that may or not be appropriate on an individual level, and discourage patients from being involved in their own care decisions, P4P is the way to go!

Avatar

Jan Krouwer

April 28th, 2009 at 6:25 am

Nevertheless, quality tools (which include quality measures) can be useful. A recent example is preventing infections in central lines (NEJM 2006;355:2725-32). Here, an infection rate was observed (the quality measure), it was studied, steps were taken, and the infection rate decreased. The quality tool relies on measuring an error rate. There are other examples such as improvement in reducing errors in anesthesiology in the late 70s and 80s.

Avatar

Michael Kirsch, M.D.

May 10th, 2009 at 3:43 pm

Every player in the game advocates for increased medical quality. How do you define medical quality and how do you measure it? You can’t. How can you assess how thorough a physician’s medical history is? How do you measure and compare a physician’s skill in examining patients? Is there a rubric that can measure a doctor’s medical judgment? These are true quality determinants in medical care. Instead, the government and others will be measuring data that has little to do with medical quality. They will measure what is easy to count because there is no way to measure what truly matters.

Michael Kirsch, M.D.
wwwMDWhistleblower.blogspot.com

Comment Form