© The Author 2007. Published by Oxford University Press.
ARTICLE |
Quality of Reporting of Cancer Prognostic Marker Studies: Association With Reported Prognostic Effect
Affiliations of authors: Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece (PAK, DDK, JPAI); Biomedical Research Institute, Foundation for Research and Technology-Hellas, Ioannina, Greece (JPAI); Institute for Clinical Research and Health Policy Studies, Department of Medicine, Tufts-New England Medical Center, Boston, MA (JPAI)
Correspondence to: John P. A. Ioannidis, MD, Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece (e-mail: jioannid{at}cc.uoi.gr).
BACKGROUND: Issues of reported study quality have not been addressed empirically with large-scale data in the cancer prognostic literature.
METHODS: Eight quality measures pertaining to study design and assay methods (i.e., blinding, prospective versus retrospective design, power calculations, outcomes' definitions, time of enrollment, reporting of variables, assay description, and assay reference) were evaluated in cancer prognostic marker studies included in meta-analyses identified in Medline and EMBASE. To be eligible, meta-analyses had to include at least six studies and to examine binary outcomes. We estimated the ratios of relative risks, which compared the overall prognostic effects (summary relative risks) between poor-quality and good-quality studies for each quality item. Between-study heterogeneity was tested with the Q statistic (statistically significant at P<.10). All statistical tests were two-sided.
RESULTS: We identified 20 meta-analyses that included 331 cancer prognostic marker studies published between 1987 and 2005. Only three (0.9%) of the 331 studies presented power calculations, 129 (39.0%) studies stated that analyses were blinded, and 73 (21.5%) stated that they were prospective. Time of enrollment was defined in 232 (70.0%), 234 (70.7%) gave lists of candidate variables, and 254 (76.7%) defined outcomes. The assay used was described in 317 (95.8%), but only 177 (53.5%) provided the assay reference. Estimates of prognostic effects from poor-quality studies varied considerably and could be larger or smaller than summary estimates derived from meta-analyses. Summary ratios of relative risks of poor- versus good-quality studies for the seven quality measures ranged from 0.95 to but 1.26, but none was statistically significantly. There was statistically significant heterogeneity (P<.10) between the ratios of relative risk estimates across meta-analyses for blinding, defining endpoints, and stating variables and assay references.
CONCLUSIONS: Among cancer prognostic marker studies, reporting quality of design and assay information often appears suboptimal, indicating that this literature may be largely unreliable. Given the potential clinical importance of prognostic marker information, improved design and reporting of these studies are warranted.
| CONTEXT AND CAVEATS Prior knowledge Many prognostic markers have been proposed for various diseases, including cancer, many have not been validated, and most have not changed clinical practice. Suboptimal study design (such as lack of blinding and retrospective design) and missing methodologic information have been associated with exaggerated results for other types of clinical research. Study design Meta-analysis of 331 studies on cancer prognostic markers from 20 meta-analyses. Contribution The body of literature for prognostic marker studies may be largely unreliable because the quality of the information presented about study design and assays used often appeared suboptimal. Implications Because of the potential clinical importance of prognostic marker information, improvements in the design and reporting of prognostic marker studies are warranted. Limitations Reported study quality is only a surrogate for true study quality. Individual meta-analyses, from which estimates and data were obtained, may themselves suffer from various biases and used different statistical measures to compare outcomes.
|
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  N. Rifai, D. G. Altman, and P. M. Bossuyt Reporting Bias in Diagnostic and Prognostic Studies: Time for Action Clin. Chem., July 1, 2008; 54(7): 1101 - 1103. [Full Text] [PDF]
|
|
|
|
 |
|
 P. A. Kyzas, E. Evangelou, D. Denaxa-Kyza, and J. P. A. Ioannidis 18F-Fluorodeoxyglucose Positron Emission Tomography to Evaluate Cervical Node Metastases in Patients With Head and Neck Squamous Cell Carcinoma: A Meta-analysis J Natl Cancer Inst, May 21, 2008; 100(10): 712 - 720. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 F. X. Real p53: It Has It All, But Will It Make It to the Clinic As a Marker in Bladder Cancer? J. Clin. Oncol., December 1, 2007; 25(34): 5341 - 5344. [Full Text] [PDF]
|
|
|
|
 |
|
 Cancer Prognostic Markers: User Beware Journal Watch Oncology and Hematology, February 16, 2007; 2007(216): 3 - 3. [Full Text]
|
|