Journal of the National Cancer Institute Advance Access originally published online on June 27, 2007
JNCI Journal of the National Cancer Institute 2007 99(13):988-990; doi:10.1093/jnci/djm057
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© Oxford University Press 2007.
NEWS |
Small Study on Industry Trial Sponsorship Leads to Big Questions About Quality and Bias
Pharmaceutical industry support has far surpassed federal funding for clinical research in the United States. And many drugs currently under development are designed by the industry to fight cancer. So researchers at the University of North Carolina School of Medicine and Dana-Farber Cancer Institute undertook a small project to see if there was any association between pharmaceutical company involvement and outcomes in breast cancer clinical trials. It was the only study of its kind conducted in breast cancer research and only the second on any type of cancer.
Published in April in Cancer, the paper looked at 140 studies conducted in 1993, 1998, and 2003—arbitrarily chosen 5-year spans—and found no statistically significant difference between positive outcomes and industry versus nonindustry authorship (78% versus 66%). But when they focused on the 56 studies published in 2003, when disclosure requirements were the tightest, they found that 84% of studies reporting pharmaceutical involvement were positive, compared with 54% of studies not associated with the industry. Industry-associated research was also more likely to be single armed (66% versus 33%) and to evaluate metastatic disease (72% versus 46%).
This is just the most recent finding in the debate over industry-sponsored trials. Supporters say companies just spend more to design good trials. But detractors say industry studies are shrouded in secrecy and may be biased. Meanwhile, researchers hope that comprehensive clinical trial registries, such a new one set up by the World Health Organization, may help eliminate some of the problems.
Similar studies in other disciplines, such as cardiology and psychiatry, have found the same associations between sponsorship and positive outcomes. But the authors of this breast cancer study weren't willing to jump to specific conclusions because their study was so limited—it just looked at published reports without the benefit of viewing any other data.
"Though it was a relatively small sample, we found a significant association between industry involvement and positive outcomes, which I think is a starting point rather than an ending point for further investigation," said Jeffrey Peppercorn, M.D., assistant professor of medicine in UNC's School of Medicine's division of hematology and oncology.
Peppercorn and the other three authors say they don't know how to explain the positive outcomes in industry-associated trials. "Possibilities to explain this could be that industry is making smarter, or perhaps safer, choices in their trial design, or that there is bias either in the study design or the reporting of results or burying of negative results, which is absolutely possible here," Peppercorn said.
"It's like if a baseball player is hitting more home runs than other baseball players. It could be because he's on steroids. But it could also be that he's a better player," he said.
The pharmaceutical industry said their good results come from old-fashioned practice that hones their trial design skills. "Companies invest a lot of time and money in their clinical research, and they spend a lot of time thinking about what to study," said Caroline Loew, Ph.D., senior vice president of science and regulatory affairs at the industry trade group, Pharmaceutical Research and Manufacturers of America (PhRMA). These studies are conducted with extreme rigor given the highly regulatory environment in which they are reviewed for approval and the intense scrutiny they receive, she said.
Peppercorn wasn't surprised by what he found, but the worldwide level of interest in the study took him aback. Newspaper articles appeared in many nations, and scientific interest was also high, he said. "I think people are hungry for an understanding of what the impact of pharmaceutical involvement is on trials and patients."
Searching for Bias
Researchers have long been trying to see through what they say is the opaque process that for-profit groups undertake when they run clinical trials, a system that many say seems to be of high quality but is not open to scrutiny. Issues that may slant research outcomes, they say, include publication bias and clinical trial designs in which the control arm agent is inferior, perhaps deliberately so. Questions about data manipulation have also arisen.
In the only other study of industry sponsorship and outcomes in cancer clinical trials, Benjamin Djulbegovic, M.D., Ph.D., a professor of medicine and oncology at the H. Lee Moffitt Cancer Center at the University of South Florida, tried to see how well industry adhered to what he said is the bedrock of clinical trials research: the uncertainty principle. The uncertainty principle is the notion that there should be substantial uncertainty about the relative value of one treatment versus another. "Otherwise, why do the trial?" he asked. "If comparator arms are unbalanced, the result could be biased."
In a 2000 study published in the Lancet, Djulbegovic looked at 136 published randomized studies of multiple myeloma agents and found that when the analysis was done according to the source of funding, studies sponsored by nonprofit organizations were almost equally split between finding new therapies to be superior to standard therapies (47% versus 53%)—which would be expected if the uncertainty principle is maintained. But randomized trials supported solely or in part by for-profit organizations were much more likely to find that new therapies were superior to standard ones (74% versus 26%).
"This likely means that preferential support was given to trials that had a greater chance of favoring one intervention over another," Djulbegovic said. In fact, they found that more industry-sponsored studies compared an innovative treatment to either a placebo or no therapy rather than comparing it to the standard of care, which is more common in studies sponsored by the government.
Djulbegovic later collaborated with researchers from other fields to look at 30 studies of the association between the outcome of research and the source of funding. The study, published in the British Medical Journal in 2003, found that systematic bias favors products that are made by the company funding the research. Possible explanations included the selection of an inappropriate comparison to the product being investigated or publication bias.
Publication bias usually means that researchers don't submit studies with negative results for publication, not the widely held view that journals don't publish them, said Kay Dickersin, Ph.D., director of the center for clinical trials at the Johns Hopkins Bloomberg School of Public Health. She said that perhaps half of the estimated 60,000 clinical trials ongoing in the United States will never be published in a scientific journal: Investigators are not submitting their results for publication. "Even if there is a bias at journals, it appears to be small compared to the bias of authors, and the majority of the problem with industry authors may be the pressure that is put on them not to publish," she said. Selective reporting of outcomes is also a problem, Dickersin added. Studies have found that at least one primary outcome between the protocol and publication changed in nearly two-thirds of clinical trials, she said.
However, academic researchers who participate in industry-sponsored studies may not know if aspects of the clinical trials they are participating in are being changed. Access to original data can be a problem in this kind of research, said Howard Brody, M.D., Ph.D., director of the institute for the medical humanities at the University of Texas Medical Branch in Galveston. "In a typical industry-funded multicenter study, the investigator at each site may see only the data from that site," said Brody, whose book Hooked: Ethics, the Medical Profession, and the Pharmaceutical Industry, was published this year. "The data will go to a central office at the company, where the company treats those data as a trade secret. There is usually no guarantee that any independent academic investigator will see the entire body of data and the decision as to how to analyze the data could be made within the company."
Another reason industry-supported studies are often positive is that companies use publications as a marketing strategy to disseminate information on a possible new use of an already approved drug, said Lisa Bero, Ph.D., a professor in the department of clinical pharmacy at the University of California, San Francisco. Then a company would not seek FDA approval for a new use but would rely on publication of these so-called positive seeding trials to get word out to physicians, who can use drugs outside approved indications.
Other design issues involve cutting off studies early or running them longer than the initial protocol called for, Bero said. "I have done studies in which I used company data to show that what was published is not exactly what was submitted to the FDA," she said. "It is very hard to understand the extent of these problems, but there is real cause to be concerned and to investigate these trials a lot more thoroughly."
No one can say that these issues are a problem specifically in oncology studies, because they haven't been examined thoroughly, Peppercorn and others say. However, the reasons for the difference in positive outcomes between industry and government or academic sponsorship in oncology clinical trials may be different from and more complex than trials in other areas because of "the different nature of the disease and the willingness of the sponsors to take more or less risk," said Jeffrey Abrams, M.D., chief of the clinical investigations branch at the National Cancer Institute.
|
"Academic/NCI trials are often riskier, multimodality trials that industry wouldn't consider and include a lot of correlative science that industry wouldn't pay for. Academic/NCI trials may compare drugs head to head, which companies don't often like to do." Abrams said.
Another difference is that "industry sponsors often target a specific type of cancer they think, based upon preclinical data, have the greatest likelihood of succeeding—and often partner with academics and NCI in disease areas other than their lead disease area," he said. "Thus, the ‘nonindustry’ trials are of higher risk in many cases and thus more likely to be negative, because there is less data going in about their likely success."
And while NCI-sponsored trials are usually looking for an improvement in outcome, industry sponsors use the FDA rules as their guide. These rules require only that a product be comparable to, not better than, an existing product, "which can set a low bar," Abrams said.
Still, Abrams thinks that companies hiding data almost inevitably get caught, so there isn't a big incentive to do that, adding that this is his own opinion since NCI hasn't studied these issues. Nevertheless, the vast sums of money involved in cancer research suggest that it is reasonable to exercise caution in interpreting outcomes in industry-sponsored trials, he said.
Registries Provide Clues
One way to provide some transparency and accountability in research is through Web-based, publicly available clinical trial registries. The idea is that these registries will provide more comprehensive disclosure of trial information, revealing studies whose results have not been published or whose outcomes have not been disclosed. That potential has led to a growing number of clinical trial registries around the world. But to date, reality has not met expectations.
For example, the National Institutes of Health has been running clinicaltrials.gov since the Web site was created by a 1997 law. The registry is a voluntary listing of clinical trials sponsored by the NIH, other federal agencies, and private industry, although registration is mandated for all drugs trials testing "serious or life-threatening diseases and conditions" conducted under FDA Investigational New Drug application regulations. As of May, it contains information on 36,249 clinical trials from approximately 140 countries. The list has grown in recent years after an FDA report found that many privately funded trials were not being registered as required. That development led to a 2005 decision by the International Committee of Medical Journal Editors not to accept articles unless the studies are registered. (JNCI adheres to these guidelines.)
But substantial issues exist within the registry, according to the registry's director, Deborah Zarin, M.D., of the National Library of Medicine. In a commentary in the May 16 issue of the Journal of the American Medical Association, Zarin wrote that without access to trial protocols, it is impossible to verify if the data entered into the registry are correct. Furthermore, there is a significant "naming problem"—a lack of clarity about an agent being tested because of concerns by companies about loss of intellectual property. Sometimes, companies use "company-specific serial numbers" that "are not tracked by any external agencies," she said.
Some experts think these registries do not go far enough because they don't require the inclusion of the trial results, published or unpublished. Several congressional bills have been introduced that would require that a written summary of the clinical trial results be made public, and the one that is farthest along is the FDA Revitalization Act of 2007, which was approved by the Senate on May 10 (a companion House bill has not yet been considered). But Zarin said that no standard format or process exists for providing this kind of information to the public or ensuring its accuracy. Many remain concerned about "relying on sponsors or other data providers with vested interests in how the results are portrayed to submit narrative summaries of results," Zarin said. "None of the policy proposals under discussion would provide registry staff with access to protocols or raw data, making the independent scientific review of database entries impossible."
The pharmaceutical industry stands by the many clinical trial registries it has set up, including one that they say contains results. Companies that belong to PhRMA have committed to disclosing the results, positive or negative, of "hypothesis testing" clinical trials on clinicalstudyresults.org, Loew said. These are not exploratory trials but statistically powered studies (mostly phase III) on drugs and biologics marketed in the U.S. or intended for marketing in the U.S., she said. The registry includes both positive and negative results and, while participation is voluntary, more than 50 companies have contributed data on 350 drugs to the registry, "which exceeds our membership," Loew said.
Dickersin said the industry's registries tend to limit the results to approved drugs, and much of the information there is already on the FDA's Web site. Zarin said these sites are generally not reviewed by experts outside the company, and one study found that the conclusions listed in these databases tended to be more favorable for the company's product than those found in published articles or FDA reviews of the same trials.
To address these problems, the World Health Organization has created a clinical trial registry that promises to further improve transparency. It launched the clinical trial search portal (http://www.who.int/trialsearch) in May. The site works as an entry point into multiple, high-quality clinical trial registries and has a global search function. Large clinical trial registries in the U.S., Australia, Britain, and Europe contribute data directly into the WHO portal, as do several pharmaceutical companies.
The idea is that any patient or investigator can search for information on a drug that has been tested in several different countries but whose results may not have been published. Even in papers that are published, "it is possible that only part of the story is told in the publication," WHO's press announcement said. "Relying on information provided only in published trial research is therefore unreliable and leads to inadequately informed treatment decisions."
The WHO portal eventually aims to provide data that no other registry yet offers, such as information on early-phase trials and complete results of all trials. However, this issue is currently being studied because "there is no formal consensus on international norms and standards for results reporting." WHO says it is also still exploring ownership, publication, and ethical issues in the reporting of clinical trial results.
Peppercorn, the researcher who raised the issue of industry-sponsored outcomes in breast cancer research, said that registries are a step toward a clearer understanding of what goes on in company-sponsored trials. Registries will help "untangle the correlations between sponsorship, research outcomes, and publication."
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||