The new issue of JAMA has a wonderful commentary – Sudden Acceleration of Diabetes Quality Measures – in which the story of recent likely harmful performance measures is outlined.
However, beginning in 2004, the National Committee for Quality Assurance (NCQA) added optimal measures for HbA1c, blood pressure, and cholesterol to the Diabetes Physician Recognition Program—HbA1c values less than 7%, blood pressure less than 130/80 mm Hg, and LDL-C values less than 100 mg/dL—which were adopted by the Bridges to Excellence pay-for-performance program.2 In 2005 the Minnesota Community Measurement Program developed a composite measure, termed the Diabetes 5 (D5), that was based on these optimal measures for HbA1c, blood pressure, and LDL-C; administration of aspirin; and nonsmoking status.3?
Despite unanimous opposition from the Technical Advisory Panel of the Alliance, the NCQA approved the optimal measure set in May 2006.4 Even though inconsistent with guidelines from nonspecialty societies, these measures were widely viewed as evidence-based standards of care because they were based on the guidelines of specialty societies that lent their leadership to the industry-sponsored public service campaign.1?
Why might this have happened? More recent studies support the Technical Advisory Panel that the "optimal measure set" has serious flaws.
This case study indicates that the quality industry's process of developing performance measures is broken and that steps need to be taken to restore accountability.
Too often we have measures based on a religious belief (e.g. lowering HgbA1c is always the proper goal) and not based upon good prospective data. Whenever we have to struggle to meet a performance goal, we run the risk of unexpected consequences.
This irresponsible process likely harmed patients. Let me repeat that sentence. This irresponsible process likely harmed patients. The reasons now are clear. Some, including the authors of this commentary complained bitterly back in 2006. We allow organizations to establish performance measures without expecting the same rigorous testing that any other intervention must have prior to approval. We would not approve a new drug without careful testing for both efficacy and safety. Should we not hold performance measurement to the same standard?
{ 3 comments… read them below or add one }
How to Ensure that Physicians Will Dislike Performance Indicators
The Veterans Health Administration jumped on the performance indicators bandwagon as soon as it started rolling. The VHA used its reasonably good electronic medical record system to remind clinicians about performance indicators such as post-MI patients receiving beta-blockers, type II diabetics without known renal disease getting urine albumin measurements at least annually, etc. Most of the performance indicators were sensible, logical, and helpful. However, whenever the JCAHO dropped some performance indicators and added new ones, the VHA immediately cancelled its automated reminder system for those that were dropped despite physicians wanting them kept in place. The VA's information technology administrators made a unilateral decision to only maintain the indicators used for federal performance measures. This bureaucratic mentality that is concerned only with hitting performance indicator targets rather than improving patient care proves that the federal government is concerned about the appearance of quality medical care rather than the actual performance of quality care.
Amen!
One has to be concerned of the influence of industry which has a clear incentive to push more rigorous standards that will require more of the pharmaceuticals they produce. Theses entities have often foisted themselves into the process of consensus guideline development by often having financial relationships with the key opinion leaders sitting on these panels. The conflict of interest is obvious to most, but apparently not to these learned thought leaders. Often, we are finding the guidelines are the speculation of benefit of these consensus panels with no significant data to support the recommendation.
I would agree that we need to have the same rigorous proof that these recommendations will prove beneficial and cost effective before insisting that physicians abide by them in order to qualify for puality bonus pools and such. If our focus becomes treating numbers and not the patient, we will not see any favorable improvement in health and outcomes resulting I fear, and possibly some detrimental effect.
As an ICU doctor, I have been told to make our VAP rate zero and indeed on paper, we will make it zero.