© The Author 2006. Published by Oxford University Press.
EDITORIAL |
What's the Rush? The Dissemination and Adoption of Preliminary Research Results
Affiliations of authors: VA Outcomes Group, White River Junction, VT, and Center for the Evaluative Clinical Sciences, Dartmouth Medical School, Hanover, NH
Correspondence to: Lisa M. Schwartz, MD, MS, Department of Veterans Affairs Medical Center, VA Outcomes Group, Mail Stop 111B, 215 N. Main St., White River Junction, VT 05009 (lisa.m.schwartz{at}dartmouth.edu).
How fast should the preliminary results of medical research be disseminated and adopted into clinical practice? In this issue of the Journal, Giordano et al. (1) provide an example of moving fast. They describe the substantial increase in use of taxane chemotherapy for women with node-positive breast cancer in the year following the presentation of the Cancer and Leukemia Group B (CALGB) 9344 study at the 1998 American Society of Clinical Oncology (ASCO) meeting—nearly 5 years before publication of the results in a peer-reviewed journal.
Moving fast is appealing. It means being on the cutting edge of medicine. It means bringing new hope to patients. There is a presumption that newer treatments are better than older ones. And all sorts of forces encourage the rapid adoption of new, so-called breakthrough treatments and technologies, including investigators with professional and financial interests, pharmaceutical companies, an uncritical news media, aggressive disease advocacy groups, and desperate patients with progressive disease and no good options.
The taxane story supports the idea that moving fast can benefit patients: Women with early breast cancer did not have to wait 5 years to get access to a useful treatment that has become a standard of care. The results presented at the 1998 ASCO meeting—released at the recommendation of the study's data safety and monitoring board (1)—came from a well-done, large (more than 3000 women), multicenter randomized trial. The investigators found a small but real benefit in a fundamentally important outcome—overall survival (2). The meeting presentation also clearly reported the associated harms (e.g., grade 3 or greater toxicities such as transient myelosuppression [21%], neuropathy [5%], and pain [5%]). Fortunately, things worked out well. The interim results in the 1998 meeting report (97% of women in the taxane plus standard chemotherapy group and 95% of those in the standard chemotherapy group were alive at 18 months) closely mirrored the final results published in 2003 (80% versus 77% were alive at 5 years) (1). But the story might have had a very different ending: The early benefit might not have held up in the longer term, and more harm might have emerged over time. The investigators and the patients were lucky.
The point is that moving fast is a gamble. When preliminary findings turn out to be true, patients benefit. When the findings are not true, patients get hurt: They are exposed to ineffective or harmful treatments, or they forgo good alternatives.
The story of gefitinib (Iressa) demonstrates the other side of the coin—a case where the adoption of preliminary results was a mistake. Unlike the taxane results, which were preliminary in time (i.e., the interim results were disseminated before publication of any results in a peer-reviewed journal article), Iressa is a story about results from a study that was preliminary in terms of design (i.e., a small, hypothesis-generating study using a surrogate outcome). Gefitinib is a drug developed for the treatment of non–small-cell lung cancer patients who failed prior chemotherapy. In 2003, the FDA approved the drug through a new accelerated process on the basis of a single uncontrolled study that found that 10% of 216 patients taking Iressa experienced a reduction in tumor size (3). Approval was granted despite concerns about important side effects, including more than 200 reports of fatal interstitial pneumonia among patients in Japan (3). Moreover, an alternative second-line therapy for non–small-cell lung cancer, taxotere, existed. It had been approved in 1999 on the basis of two randomized trials' showing increased survival (4). There was no strong case for the approval of Iressa based on these early data. Nevertheless, more than 200 000 people worldwide used the drug by 2004 (5). As a condition for the accelerated approval of Iressa, FDA required the manufacturer, AstraZeneca, to demonstrate a survival benefit in a subsequent clinical trial. A placebo-controlled randomized trial of 1700 patients was conducted but failed to show such a benefit. Consequently, in 2005, FDA restricted the use of Iressa to existing or previous users and patients in clinical trials (6).
Ideally, physicians would never have to rely on preliminary research; they would have timely access to a complete report published in a peer-reviewed journal or to the results of definitive clinical trials. But in reality, preliminary research—and its enthusiastic dissemination—is here to stay.
Fortunately, there are ways to improve the dissemination of preliminary results. One way is for the organizers of scientific meetings to make more complete information available by posting on their Web site the actual meeting presentations, not just the published abstracts. Indeed, as mentioned by Giordano et al. (7), ASCO has done so since 1999. Another approach is for meeting organizers to establish the expectation that major presentations be published at (or near) the time of the meeting. The nearly 5 years between presentation of the interim results of the CALGB 9344 trial at the ASCO meeting and publication of a full report in a peer-reviewed journal is unreasonable. If interim results are important enough to act on, a peer-reviewed publication is necessary. Because meeting organizers and medical journal editors share a common interest in getting important research out to the larger scientific community, it makes sense for them to work together to coordinate the presentation, review, and publication processes.
Furthermore, to routinely highlight the cautions about making inferences from intrinsically limited studies, meeting and journal abstracts (as well as the accompanying press releases) should include a "Caution" or "Limitation" header. Findings presented at scientific meetings may change substantially by the time they are published in a peer-reviewed journal, and many may never be published at all (8,9). Clear and prominent cautions would serve not only clinicians but the news media as well. Although news stories about preliminary research are common, they rarely note important cautions, and the public may consequently be misled about the validity and relevance of the science (10).
Regardless of what happens on the dissemination side, physicians need to approach preliminary research cautiously. Figure 1 provides a series of questions to help clinicians decide whether to encourage patients to gamble on preliminary results. First, did the research show an important clinical benefit? By "clinical benefit," we mean interventions that affect how patients feel or function or whether they survive, as opposed to surrogate outcomes such as a change in tumor size or lab findings. The patient's clinical state matters a lot. Patients who feel well or have a good prognosis already (i.e., with standard treatment) should demand more benefit than patients who are sick or dying. Second, are there important downsides to using the new regimen? Judgments about benefit need to be made in the context of harm. Important side effects may be acceptable in exchange for a clinically meaningful benefit. But even small harms may outweigh any benefit in a surrogate outcome. Third, how strong is the evidence for the findings? Results from large, long-term randomized trials are more compelling than results from small trials or controlled observational studies and much more so than the results of a case series. Fourth, how does the new research fit with prior work? Contradictory findings are generally an indication for caution; however, the first good randomized trial data may be sufficient to trump a body of observational work [as occurred with regard to hormone replacement therapy with the publication of the Women's Health Initiative trial (11)]. Finally, do effective treatment alternatives exist? It makes little sense to risk exposure to a new, therapeutically equivalent drug when safe and effective alternatives with longer track records are available.
|
Physicians are confronted with preliminary research findings all the time. To decide whether the findings are good enough to change practice, they must be able to answer some fundamental questions. The most basic question, of course, is what is the rush?
NOTES
The authors contributed equally to the creation of this report. The order of their names is entirely arbitrary.
Drs. Woloshin and Schwartz were supported by Robert Wood Johnson Generalist Faculty Scholar Awards and by a Research Enhancement Award from the Department of Veterans Affairs. The views expressed herein do not necessarily represent the views of the Department of Veterans Affairs or the United States Government.
We thank H. Gilbert Welch, MD, MPH, and Alexander Kallen, MD, for their helpful reviews of this editorial.
REFERENCES
(1) Henderson I, Berry D, Demetri G, Cirrincione C, Goldstein L, Martino S, et al. Improved outcomes from adding sequential paclitaxel but not from escalating doxorubicin dose in an adjuvant chemotherapy regimen for patients with node-positive primary breast cancer. J Clin Oncol 2003;21:976–83.
(2) Henderson I, Berry D, Demetri G, Cirrincione C, Goldstein L, Martino S, et al. Improved disease-free and overall survival from the addition of sequential paclitaxel but not from the escalation of doxorubicin dose in the adjuvant chemotherapy of patients with node-positive primary breast cancer. Proc ASCO 1998;17:101a.
(3) Pollack A. Drug's approval hints at flexibility in F.D.A. process. New York Times 2003 May 6; Sect. 1.
(4) United States Food and Drug Administration. FDA Talk Paper: FDA approves new indication for Taxotere. Available at http://www.fda.gov/bbs/topics/ANSWERS/ANS00991.html. [Last accessed: January 31, 2006.]
(5) Pollack A. Lung-cancer drug shows unfavorable trial results. New York Times 2004 December 20; Sect. 2.
(6) United States Food and Drug Administration. Questions and Answers on Iressa (gefitinib). Available at http://www.fda.gov/cder/drug/infopage/iressa/iressaQ&A2005.htm. [Last accessed: January 31, 2006.]
(7) Giordano SH, Duan Z, Kuo Y-F, Hortobagyi GN, Freeman J, Goodwin JS. Impact of a scientific presentation on community treatment patterns for primary breast cancer. J Natl Cancer Inst 2006;98:382–8.
(8) Schwartz L, Woloshin S, Baczek L. Media coverage of scientific meetings: too much, too soon? JAMA 2002;287:2859–63.
(9) Scherer R, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. JAMA 1994;272:158–62.
(10) Woloshin S, Schwartz L. Media reporting on research presented at scientific meetings: more caution needed. Abstract presented at the Fifth International Congress on Peer Review and Biomedical Publication held in Chicago, IL in September 2005.
(11) Writing Group for the Women's Health Initiative. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results for the Women's Health Initiative randomized controlled trial. JAMA 2002;288:321–33.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||