Comparative Effectiveness Research

Healthcare has never been more enmeshed in the political moment.  Ever since the announcement of Don Berwick to lead CMS, opposition centered around Berwick's supposed desire to ration care.  Berwick's supporters argue that care is already rationed arbitrarily by insurers.  And it's not credible to paint Berwick as favoring government rationing.   Berwick advocates use of evidence-based medicine and comparative effectiveness research (CER) to more effectively balance outcomes and costs.

Comparative effectiveness research evaluates methods to “prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels." (IOM 2009).

Mathematica Policy Research's recent report on the policy implications of comparative effectiveness research found that "interest in evaluating which health care interventions work best under what circumstances has surged in recent years as policymakers seek tools to moderate the cost of public entitlement programs and to facilitate affordable coverage expansions in an era of rising health spending. Although recent legislation has taken positions on some of the fundamental issues relating to expanded public support for comparative effectiveness research (CER), a number of key political and policy questions have been deferred to another time and venue. Thus, continuing disagreements may pose challenges to the new CER initiative." The key policy questions include: 1) financing CER, 2) directing CER, 3) using CER appropriately and 4) defining CER.

The Affordable Care Act (ACA) "focused the research on developing information of the greatest relevance to clinicians, precluding research focused on cost comparisons, especially cost-effectiveness analysis. Similarly, ACA includes explicit prohibitions on the use of CER funds to calculate so-called quality-adjusted life years, used in assessing the relative value of interventions. The ACA stance reflects a decision to avoid conflating research findings about the relative effectiveness of alternative approaches with judgments about the value of any incremental improvements found. Such information on marginal cost-effectiveness has proven meaningful in public efforts in which the citizens accept the legitimacy of restricting access because of cost considerations; but the recent debate suggests the U.S. public is not ready. Perhaps continued pressure from health cost growth will change the public perspective, but for the present, cost-effectiveness evaluation will not be part of ACA-financed CER."

So what does comparative effectiveness research look like?  It's quite distant from the stuff of political pundits.  It's dry.  It's scientific.  And it's not always conclusive.

The Agency for Healthcare Research and Quality(AHRQ) recently published its report on the Comparative Effectiveness of Nonoperative and Operative Treatments for Rotator Cuff Tears.  It's not easy to understand whether surgery  or non-operative interventions worked best.  The strength of evidence for most findings is low, with a few "moderates".

Only limited conclusions can be reached - certainly not sufficient for the thousands of patients making decisions each year for rotator cuff surgery. 'For the majority of interventions, only sparse data are available, precluding firm conclusions for any single approach or for the optimal overall management of this condition. The paucity of evidence related to early versus delayed surgery is of particular concern, as patients and providers must decide whether to attempt initial nonoperative management or proceed immediately with surgical repair. The majority of the data is derived from studies of low methodological quality or from study designs associated with higher risk of bias (e.g., observational and before-and-after studies). Overall, the evidence shows that all interventions result in substantial improvements; however, few differences of clinical importance are evident when comparisons between interventions are available. Complication rates were generally low and the majority of complications were not deemed to be clinically importa  nt; therefore, the benefit of receiving treatment for rotator cuff tears appears to outweigh the risk of associated harms. Future research is needed to determine the relative effectiveness of rotator cuff treatment options."

In many examples like this, data is the key deficit both in terms of statistically adequate samples and data quality.

Here's AHRQ's Effective Health Care P:rogram list of reviews in progress: