Value-Based Medicine

Study Results: Going Behind the Numbers

By Melissa M. Brown, MD, MN, MBA

I find it hard to recall what I learned yesterday, much less last month. So let's quickly recap what we've learned about Value-Based Medicine in my first two columns: (1) value refers to improvement of length-of-life and/or quality-of-life and most often the latter in ophthalmology; (2) the foundation blocks for Value-Based Medicine are medical evidence, value to patients (comparative effectiveness) and the amalgamation of relevant costs (cost-effectiveness); (3) the "p-level" defines the chance that a clinical study supports an effect that truly doesn't exist and the "power" defines the chance that the study will detect a true effect with the number of patients studied. The power is especially important when there is no effect seen (the negative study).

Judging Validity of Study Results

Now what? Before searching for the plevel, take a moment to assess the validity of any study. How believable are the results?

There are five hurdles to clear: (1) How was treatment allocated — randomly or not? Random allocation clears the hurdle with greater ease; (2) Were all enrolled patients accounted for at the end of the study? If greater than 20% of patients were lost to follow-up during the study, then suspect a glitch; (3) Was the treatment masked to patient and doctor? Better not to know at the time of study so as to influence the data; (4) Other than the actual treatment, were the study groups treated identically in all respects?; and (5) Were the groups initially similar at the start of the study? And if they were dissimilar, did the researchers account for the potentially confounding baseline variables?

Assess the significance of the study, both clinically and statistically. Did the researchers seek a clinically relevant difference and were there enough patients to detect this difference?

This evaluation uses the experience of clinicians. Statistical significance can be assessed by looking at the chance of making a Type 1 error, the p-level — the chance of utilizing an intervention that, in actuality, demonstrates no difference from a placebo or sham intervention.

Lastly, let's consider how we gauge the importance of the randomized clinical trial (RCT). This brings to the fore an important but often confusing consideration: risk reduction.

Figures Can Be Misleading

It is important to understand the difference between relative and absolute measures of outcomes.

Take, for instance, the natural history of a sample disease. Assume that without a new treatment, 5% of people will experience a bad outcome. However, with the new treatment, the study in question supports that only 3% of treated people will have a bad outcome. The relative risk (% saved of those who would have otherwise obtained a bad outcome without the treatment) has been reduced by twofifths, or 40%. However in absolute terms, the % risk of a bad outcome is reduced from 5% to 3%, or 2%. Hmm… 40% vs. 2% reduction of risk can be very misleading, despite a report being published in a peer review journal with all its incumbent cachet. Beware of the studies that describe "risk reduction" without identifying absolute risk reduction (ARR) or relative risk reduction (RRR). Clinically, we want to consider the absolute risk reduction to evaluate the importance to our patients. The number of patients needed to treat (NNT) is the inverse of the absolute risk reduction. In this case 1/0.02 or 50 patients are needed to be treated to obtain one therapeutic success.

Evaluate Study Results Critically

To use the evidence presented to us from a variety of avenues, whether peer-reviewed studies, specialty publications or company-provided literature, be a critical consumer of the evidence. Carefully review the clinical and statistical evaluations and seek to use interventions that maximize the "absolute risk" reductions. Using efficacious interventions that treat patients who have the potential for significant morbidity will do exactly that.

In ophthalmology, we strive to improve the quality-of-life of our patients. In upcoming columns, we will discuss how we can measure quality-of-life improvement, reduce the burden of illness and increase costeffectiveness to our patients, their families, our communities and society in general. OM

Melissa M. Brown, MD, MN, MBA, is president and CEO of the Center for Value-Based Medicine in Philadelphia, She can be reached via e-mail at mbrown@valuebasedmedicine.com