| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© The Author 2007. Published by Oxford University Press.
ARTICLE |
Critical Review of Published Microarray Studies for Cancer Outcome and Guidelines on Statistical Analysis and Reporting
Affiliations of authors: Biometric Research Branch, Division of Cancer Treatment and Diagnosis, National Cancer Institute, National Institutes of Health, Bethesda, MD (AD, RMS); Université Paris VII Denis Diderot, Paris, France (AD); Assistance Publique-Hôpitaux de Paris, Service de Dermatologie, Hôpital Saint-Louis, Paris, France (AD)
Correspondence to: Richard M. Simon, DSc, National Cancer Institute, 9000 Rockville Pike, MSC 7434, Bethesda, MD 20892 (e-mail: rsimon{at}nih.gov).
BACKGROUND: Both the validity and the reproducibility of microarray-based clinical research have been challenged. There is a need for critical review of the statistical analysis and reporting in published microarray studies that focus on cancer-related clinical outcomes.
METHODS: Studies published through 2004 in which microarray-based gene expression profiles were analyzed for their relation to a clinical cancer outcome were identified through a Medline search followed by hand screening of abstracts and full text articles. Studies that were eligible for our analysis addressed one or more outcomes that were either an event occurring during follow-up, such as death or relapse, or a therapeutic response. We recorded descriptive characteristics for all the selected studies. A critical review of outcome-related statistical analyses was undertaken for the articles published in 2004.
RESULTS: Ninety studies were identified, and their descriptive characteristics are presented. Sixty-eight (76%) were published in journals of impact factor greater than 6. A detailed account of the 42 studies (47%) published in 2004 is reported. Twenty-one (50%) of them contained at least one of the following three basic flaws: 1) in outcome-related gene finding, an unstated, unclear, or inadequate control for multiple testing; 2) in class discovery, a spurious claim of correlation between clusters and clinical outcome, made after clustering samples using a selection of outcome-related differentially expressed genes; or 3) in supervised prediction, a biased estimation of the prediction accuracy through an incorrect cross-validation procedure.
CONCLUSIONS: The most common and serious mistakes and misunderstandings recorded in published studies are described and illustrated. Based on this analysis, a proposal of guidelines for statistical analysis and reporting for clinical microarray studies, presented as a checklist of "Do's and Don'ts," is provided.
| CONTEXT AND CAVEATS Prior knowledge The use of microarray technology has generated great excitement for its potential to identify biomarkers for cancer outcomes, but the reproducibility and validity of findings based on microarray data have come under widespread challenge. Study design This is a systematic review of microarray studies in which gene expression data were analyzed for relationships with cancer outcomes. Contribution Common methodologic errors committed in statistical analysis of the relationship of gene expression data to cancer outcomes were identified and explained. A set of useable guidelines for statistical analysis and reporting of clinical microarray studies were created for the cancer research community. Implications The new guidelines could serve as an accessible and common basis for discussion among all cancer researchers involved in microarray investigations. Limitations Technical procedures for generating reproducible gene expression data are not addressed here.
|
CiteULike 
Connotea 
Del.icio.us What's this?
Related Article in JNCI
- Press Release: Study Identifies Common Flaws in Oncology Microarray Studies
- Andrea Widener
J Natl Cancer Inst 2007 99: 97.[Extract] [Full Text]
This article has been cited by other articles:
|
|
 |
|
  Z. Tian, T. Hwang, and R. Kuang A hypergraph-based learning algorithm for classifying gene expression and arrayCGH data with prior knowledge Bioinformatics, November 1, 2009; 25(21): 2831 - 2838. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. H. Dairkee, A. Sayeed, G. Luciani, S. Champion, Z. Meng, L. R. Jakkula, H. S. Feiler, J. W. Gray, and D. H. Moore Immutable Functional Attributes of Histologic Grade Revealed by Context-Independent Gene Expression in Primary Breast Cancer Cells Cancer Res., October 1, 2009; 69(19): 7826 - 7834. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Baek, C.-A. Tsai, and J. J. Chen Development of biomarker classifiers from high-dimensional data Brief Bioinform, September 1, 2009; 10(5): 537 - 546. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Sotiriou and L. Pusztai Gene-Expression Signatures in Breast Cancer N. Engl. J. Med., February 19, 2009; 360(8): 790 - 800. [Full Text] [PDF]
|
|
|
|
 |
|
 B. J. Wouters, B. Lowenberg, and R. Delwel A decade of genome-wide gene expression profiling in acute myeloid leukemia: flashback and prospects Blood, January 8, 2009; 113(2): 291 - 298. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. A. Troester, R. C. Millikan, and C. M. Perou Microarrays and Epidemiology: Ensuring the Impact and Accessibility of Research Findings Cancer Epidemiol. Biomarkers Prev., January 1, 2009; 18(1): 1 - 4. [Full Text] [PDF]
|
|
|
|
 |
|
 M. J. Duffy and J. Crown A Personalized Approach to Cancer Treatment: How Biomarkers Can Help Clin. Chem., November 1, 2008; 54(11): 1770 - 1779. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. V. Ballman Genetics and Genomics: Gene Expression Microarrays Circulation, October 7, 2008; 118(15): 1593 - 1597. [Full Text] [PDF]
|
|
|
|
 |
|
 R. Simon The Use of Genomics in Clinical Trial Design Clin. Cancer Res., October 1, 2008; 14(19): 5984 - 5993. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. M. Sturgeon, B. R. Hoffman, D. W. Chan, S.-L. Ch'ng, E. Hammond, D. F. Hayes, L. A. Liotta, E. F. Petricoin, M. Schmitt, O. J. Semmes, et al. National Academy of Clinical Biochemistry Laboratory Medicine Practice Guidelines for Use of Tumor Markers in Clinical Practice: Quality Requirements Clin. Chem., August 1, 2008; 54(8): e1 - e10. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Bonome, D. A. Levine, J. Shih, M. Randonovich, C. A. Pise-Masison, F. Bogomolniy, L. Ozbun, J. Brady, J. C. Barrett, J. Boyd, et al. A Gene Signature Predicting for Survival in Suboptimally Debulked Patients with Ovarian Cancer Cancer Res., July 1, 2008; 68(13): 5478 - 5486. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Z. Sun, D. A. Wigle, and P. Yang Non-Overlapping and Non-Cell-Type-Specific Gene Expression Signatures Predict Lung Cancer Survival J. Clin. Oncol., February 20, 2008; 26(6): 877 - 883. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. L. Friedman Hepatic Stellate Cells: Protean, Multifunctional, and Enigmatic Cells of the Liver Physiol Rev, January 1, 2008; 88(1): 125 - 172. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. K. Lau, P. C. Boutros, M. Pintilie, F. H. Blackhall, C.-Q. Zhu, D. Strumpf, M. R. Johnston, G. Darling, S. Keshavjee, T. K. Waddell, et al. Three-Gene Prognostic Classifier for Early-Stage Non Small-Cell Lung Cancer J. Clin. Oncol., December 10, 2007; 25(35): 5562 - 5569. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A.-N. Spiess, C. Feig, W. Schulze, F. Chalmel, H. Cappallo-Obermann, M. Primig, and C. Kirchhoff Cross-platform gene expression signature of human spermatogenic failure reveals inflammatory-like response Hum. Reprod., November 1, 2007; 22(11): 2936 - 2946. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Vekony, B. Ylstra, S. M. Wilting, G. A. Meijer, M. A. van de Wiel, C. R. Leemans, I. van der Waal, and E. Bloemena DNA Copy Number Gains at Loci of Growth Factors and Their Receptors in Salivary Gland Adenoid Cystic Carcinoma Clin. Cancer Res., June 1, 2007; 13(11): 3133 - 3139. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Diez, R. Alvarez, and A. Dopazo Codelink: an R package for analysis of GE healthcare gene expression bioarrays Bioinformatics, May 1, 2007; 23(9): 1168 - 1169. [Abstract] [Full Text] [PDF]
|
|