Performance indicators need testing

14 Oct
2009

Readers know my skepticism concerning performance indicators. My skepticism turns to antagonism when performance indicators are used for P4P or report cards. Why am I so skeptic? That is the focus of this rant.

We cannot begin using a new drug until that drug has shown efficacy. We require extensive testing for both efficacy and risks.

Performance indicators are prescribed treatments. Yet we have no requirement that promoters of performance indicators test their indicators prior to insisting that we adopt their measure.

Recent examples illustrate my point. We all know the 4 hr pneumonia debacle. Researchers searched databases and learned that pneumonia patients who had antibiotics started within 4 hours of emergency department arrival had better outcomes. So we quickly introduced incentives to hospitals based on the percentage of pneumonia patients receiving antibiotics within 4 hours.

After introduction of this new treatment standard, we learned 2 important things. When you provide incentives for early antibiotics, many patients without pneumonia received unnecessary antibiotics.  Then careful research showed that the patients not receiving antibiotics within 4 hours had more complex and less clear presentations.

We have a classic debate over HgbA1c goals.  In a recent rant, I discussed research showing that achieving HgbA1c < 7 in CHF patients led to higher mortality.  This performance measure may well injure patients.

Those who promote performance indicators without prospectively understanding the impact of those indicators do a disservice to our patients.  In medicine we have a long history of the dangers of untested expert opinion.  Experts have biases and do not always develop the best guidelines and subsequently performance measures.

They have promoted the idea that performance indicators measure quality.  They have promoted the idea that following their performance measures will improve outcomes.  Show me the evidence.

Recall the adjustment in Hgb targets for erythropoietin therapy.  Remember when beta blockers were contraindicated in heart failure.  Remember using estrogens in post-menopausal females to prevent heart disease.  Remember that all ideas in medicine should require testing to understand the risks and benefits.

Expert panels may have excellent ideas, or they may not.  We must adopt the philosophy of Missouri – SHOW ME!

Related posts:

  1. Performance measure overdose
  2. Measurement – the good and bad
  3. Guidelines and performance measures need testing
  4. Knowing it ain’t the same as doing it
  5. My position on performance measures

Related posts brought to you by Yet Another Related Posts Plugin.

5 Responses to Performance indicators need testing

Avatar

Performance indicators need testing | HEALTH INFO CENTER

October 14th, 2009 at 7:05 am

[...] Readers know my doubt connected with opening indicators. My doubt turns to enmity when opening indicators have been used for P4P or inform cards. Why am you so skeptic? That is the concentration of this rant. We cannot proceed regulating the brand new drug until which drug has shown efficacy. We need endless contrast for both efficiency as well as risks. Performance indicators have been prescribed treatments. Yet you have no order which promoters of opening indicators exam their indicators before to insi Blog Source [...]

Avatar

clarification

October 14th, 2009 at 12:23 pm

You mean the _use_ of performance indicators needs testing, right? Especially their “high-stakes” use.

The indicators themselves have no impact on patient care. It’s their use that has an impact. Clearer language produces clearer thoughts.

On this whole 4-hour antibiotic timing thing, the published literature don’t paint a clear picture of effects on patient care, for better or worse.

Avatar

pcb

October 15th, 2009 at 8:30 am

db,

it seems your rant is really about efficacy vs. effectiveness on one hand, and the ever changing world of medical knowledge on the other.

Of course medical knowledge changes, as you point out with beta blockers, estrogen, etc. that doesn’t say much about whether p4p guidelines are good or not, it just says they need to be flexible and able to change with our knowledge.

The efficacy vs. effectiveness argument is more compelling and gets to the heart of the matter. Are carefully controlled trials in selected populations a sound foundation for designing guidelines applied to the population as a whole? How much do we trust a trials’ design and results, especially when sponsored? Were unintended consequences properly considered? How important is the clinical benefit in the trial? How does my patient differ from those studied in the trial? How do I include patient preferences and values in the discussion? What do I prioritize if there are several guidelines to consider?

These are complex decisons to be made at the individual patient level. There may be algorithm basesd guidelines for diseases, but there are none for patients, which is why it’s called the art of medicine.

Avatar

PookieMD

October 18th, 2009 at 9:11 am

DB, excellent rant! I have to agree with PCB though as well–many patients are much more complicated than the test population that is used to make conclusions and devlop guideines for care (like the A1-C controversy.) The key is that P4P etc is based on a very narrow population and marker for disease–and lots of patients don’t fit that population and the guidelines! But right now, we as physicians are in a position of justifying why a patient was not treated exactly with in the guidelines–as you point out with the pneumonia-antibiotic unanticipated outcomes! Keep ‘em coming!

Avatar

justsomedoc

October 18th, 2009 at 4:05 pm

The problem is that performance indicators are chosen because they are easy to measure, not because they truely indicate quality.

All they do is drive physicians to practice to meet numbers whether or not they make clinical sense.

Comment Form