Appropriately Evaluated Data Can Lead to Improved Outcomes

Observational studies vs. randomized controlled clinical trial outcomes.

BY JOHN R. WITTPENN, M.D.

With the bounty of data harvested from the Association for Research in Vision and Ophthalmology (ARVO) meeting each spring, now is a particularly fitting time for ophthalmologists to acknowledge the importance of personally separating the wheat from the chaff.

Randomized controlled clinical trials are considered the most reliable source of information in the medical community, but post-marketing, observational studies abound because they too play a critical part in identifying, developing and bringing to market the therapeutics that improve the quality of life for our patients. Because of the pivotal role that both types of studies play in research and development, it is not uncommon to hear comparisons of outcomes of these divergent data analysis methods. Yet pitting observational studies against randomized controlled clinical trials is essentially like comparing apples to oranges.

All data have value, but the clinician must look at it through the appropriate lens to be able to see it clearly. Understanding the strengths and weaknesses of these study designs enables clinicians to make the most of the data reported. This article will outline the differences between observational studies and randomized controlled clinical trials, highlighting key distinctions and detailing their strengths and limitations.

Back to Basics

Observational studies interpret and report what happens when patients are exposed to an intervention. These studies get their name from the basic fact that findings are derived from observations of correlations between administered treatments and patient response. These studies can be prospective, based on a future evaluation of a set number of cases at a designated endpoint, or they can be retrospective, which is essentially a review of cases that were done in the past.

Observational studies are relatively easy and inexpensive to perform, and they play an important role in clinical research in that they provide preliminary evidence that can be used as the basis for hypotheses in stronger controlled clinical trials. However, their informal design predisposes them to bias and make it cumbersome to confirm if their outcomes are due to the hypothesized treatment effect or to an alternative explanation.

Some observational studies, but certainly not all, include smaller numbers of participants. This shortcoming can undermine a study's ability to report statistically significant findings, which refers to the likelihood that the study outcome is a result of something other than just chance. A clear exception would be retrospective studies of practice patterns or environmental effects coordinated across multiple sites over many years.

An example of a recent observational study is one where my colleagues and I evaluated the use of topical cyclosporine 0.05% for the treatment of chronic active ocular rosacea.¹ I found that topical Restasis (Allergan) resulted in significant improvement of signs and symptoms in 80% of the patients within 6 months. This report on Restasis suggests that it is an effective treatment for ocular rosacea but does not prove it. Clearly, observational studies are limited by the potential for bias.

The Value of Clinical Trials

Clinical studies, on the other hand, seek to answer definitively whether or not an intervention is beneficial. In these studies, patients are typically grouped randomly — or randomized — into either a treatment group or a control group. The control group receives either the current standard treatment or placebo; and the treatment group receives the product being evaluated. The results of the control group are then compared with those of the treatment group.

These randomized clinical studies seek to remove bias as much as possible by masking the investigator, and in some instances even the patient, as to the type of treatment being used. Single-blinding means the patient doesn't know if he or she is receiving the experimental product, an established treatment or a placebo — but the clinician does. Clinical trials are usually double-blinded, which means that neither the participant nor the clinician knows during the trial which participants are receiving the experimental product.

Randomized controlled clinical trials often involve multiple study centers and a greater number of patients than observational studies do. The number of patients enrolled in a study has a large impact on the study's ability to reliably detect the effect of the intervention. The larger the sample size, or number of participants in the trial, the greater the statistical power. The number of patients enrolled in a study also dictates how costly and complicated it is to accomplish. Culling large amounts of data from large numbers of patients requires administrative personnel to coordinate patient participation as well as clinical personnel to analyze the data, which makes these trials far more expensive to carry out than observational studies. Careful monitoring — often by an independent investigator — also adds to the expense associated with randomized clinical trials.

A noteworthy example of a randomized controlled clinical trial is one by Kammer and colleagues evaluating the effect of switching prostaglandin analogues (PGA) in patients who require additional IOP lowering.² This investigator-masked, prospective, 17-center study evaluated the safety and efficacy of switching Xalatan (Pfizer) patients who needed additional IOP lowering to either Lumigan (Allergan) or Travatan (Alcon). After 3 months, the investigators found that Lumigan provided greater mean diurnal IOP lowering than Travatan in patients on Xalatan requiring additional IOP lowering, and that additional IOP lowering was observed with Lumigan, even at relatively low Xalatan baselines. They concluded that longer follow-up was needed to further evaluate the effects of switching PGAs. Masking, randomization and its multicenter design underscore the value of this study's findings, at the initial 3-month evaluation interval, as well as when further findings are reported.

Another example of a well-designed clinical trial is the ACME study, which involved 14 study centers and 546 subjects.³ The ACME study evaluated the use of Acular LS (Allergan) along with steroids to prevent CME in cataract patients with no known risk factors for CME. Patients were randomized to receive either perioperative Acular LS and steroid or steroid alone at the time of cataract surgery. OCT was performed pre-operatively and then again at 4 weeks post-operatively. The digitized images were then analyzed by a retinal specialist and classified according to degree of CME. None of the patients receiving Acular LS were found to have CME as compared to 2.4% of patients receiving steroids alone, which was statistically significant. This type of study conclusively demonstrates that ketorolac, in cataract patients with no known risk for CME, significantly reduces and may eliminate the risk of CME.

Observational studies are valuable in identifying treatment-effect correlations that may be worthy of further exploration in randomized controlled clinical trials. However, their findings can be limited by the absence of masking and randomization. Randomized controlled clinical trials, on the other hand, are designed to definitively answer the questions that are asked. The use of control groups and masking eliminate bias, while large numbers of participants increase the ability to produce quantifiable, statistically significant results.

Keeping abreast of outcomes of both randomized controlled clinical trials and observational studies is recommended so that clinicians can factor in all available information when making treatment decisions. Making treatment decisions on comparisons of these disparate studies, however, is counterintuitive. It is critical that observational studies and randomized clinical trials not be held up next to one another for comparison — and if corporate sponsors or medical journals do — then it is the physician's duty to consider the findings with caution. OM

References

1. Perry HD, Wittpenn JR, D'Aversa G, Donnenfeld ED. Topical cyclosporine 0.05% for the treatment of chronic, active ocular rosacea. Presented at: the Annual Meeting of the Association for Research in Vision and Ophthalmology (ARVO); May 1-5, 2005; Fort Lauderdale, Fla.

2. J. A. Kammer; B. Katzman; S. L. Ackerman; D. Hollander. Effect of Bimatoprost 0.03% Vs. Travoprost 0.004% in Patients on Latanoprost 0.005% Requiring Additional IOP Lowering. Presented at: the American Society of Cataract and Refractive Surgery (ASCRS) Symposium; April 4 to 9, 2008; Chicago, Ill.

3. Wittpenn, JR, Silverstein, S, Hunkeler, J, Kenyon, K. ACME Study Group. Subclinical Cystoid Macular Edema (CME) Reduces Contrast Sensitivity and Final Visual Acuity in Low-Risk Cataract Patients. Presented at: the Annual Meeting of the Association for Research in Vision and Ophthalmology (ARVO); May 6-10, 2007; Fort Lauderdale, Fla.

John R. Wittpenn, M.D., is associate clinical professor, department of Ophthalmology, State University of New York, Stony Brook, and partner, Ophthalmic Consultants of Long Island, Rockville Centre, N.Y. He has no financial interest. He is a consultant to Allergan and on their speakers bureau.