Journal of the National Cancer Institute Advance Access originally published online on February 12, 2008
JNCI Journal of the National Cancer Institute 2008 100(4):284; doi:10.1093/jnci/djm309
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© The Author 2008. Published by Oxford University Press.
CORRESPONDENCE |
Re: Low-Fat Dietary Pattern and Cancer Incidence in the Women's Health Initiative Dietary Modification Randomized Controlled Trial
Affiliation of author: National Institute of Statistical Sciences, Research Triangle Park, NC
Correspondence to: S. Stanley Young, PhD, National Institute of Statistical Sciences, PO Box 14006, Research Triangle Park, NC 27709 (e-mail: young{at}niss.org).
Making multiple claims from one data set requires some care with the statistical analysis; otherwise, scientists making the claims and their readers might easily be fooled by randomness. In the Woman's Health Initiative Dietary Modification Trial, two primary questions related to a low-fat vs a normal diet and cancer risk were examined (1–2). These hypotheses were motivated by extensive epidemiology work and were expected to be true based on considerable evidence. In fact, neither of these claims was validated in the randomized trial. That these claims were not validated surprised some people, but the results should have been expected; Ioannidis (3) pointed out that claims from nonrandomized medical studies fail to replicate approximately 80% of the time. One reason (4) that claims coming from epidemiology studies fail to replicate is that epidemiologists routinely do not correct their statistical analysis for multiple testing (5).
One of the simplest ways to correct for multiple testing is to use the Bonferroni correction, whereby the number of questions under consideration is counted and the unadjusted P values are multiplied by the number of questions. More sophisticated methods are available [(6) among many others]. Of course, the data analysis strategy and the counting should be specified before the data are examined.
Counting the questions Prentice et al. (7) have under consideration is a bit elusive. There are two named primary questions, breast and colorectal cancers. A normal clinical trial strategy would be to multiply each by two to give adjusted P values; any possible effects on secondary endpoints would need to be verified in a new study. In fact, five types of cancer are under consideration: breast, colon, rectum, ovary, and endometrium, as are three time periods: the first 4 years, the next 4.1 years, and total, giving a Bonferroni multiplier of 15. There are more than 60 P values reported in tables 1–5, and none are adjusted for multiple comparisons. It is possible that many additional P values were computed but are not reported. The three smallest P values are .03, .05, and .05. Any reasonable adjustment renders these P values non–statistically significant; seeing a P value as small as .03 is quite likely by chance alone. In clinical trials, results must be statistically significant in two studies to make a claim, and the usual convention of science is that anyone wishing to assert a claim should have strong supporting evidence. Although technically correct, the summary statement "A low-fat dietary pattern may reduce ..." could easily be misinterpreted. More accurate would be the statement, "Epidemiology studies suggested that five types of cancer might be reduced by adhering to a low-fat diet. A well-conducted, large, randomized trial gives no statistical support to any of these claims."
REFERENCES
1. Prentice RL, Caan B, Chlebowski RT, et al. Low-fat dietary pattern and risk of invasive breast cancer: the Women's Health Initiative Randomized Controlled Dietary Modification Trial. In: JAMA (2006) (6):629–642.
2. Beresford SA, Johnson KC, Ritenbaugh C, et al. Low-fat dietary pattern and risk of colorectal cancer. the Women's Health Initiative Randomized Controlled Dietary Modification Trial. In: JAMA (2006) (6):643–654.
3. Ioannidis JPA. Contradicted and initially stronger effects in highly cited clinical research. JAMA (2005) 294(2):218–228.
4. Austin PC, Mamdani MM, Juurlink DN, Hux JE. Testing multiple statistical hypotheses resulted in spurious associations: a study of astrological signs and health. J Clin Epidemiol (2006) 59(9):964–969.[CrossRef][Web of Science][Medline]
5. Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology (1990) 1(1):43–46.[Medline]
6. Westfall PH, Young SS. Resampling-Based Multiple Testing (1993) New York: Wiley.
7. Prentice RL, Thomson CA, Caan B, et al. Low-fat dietary pattern and cancer incidence in the Women's Health Initiative Dietary Modification Randomized Controlled Trial. J Natl Cancer Inst (2007) 99(20):1534–1543.
Related Article in JNCI
Response to this Correspondence
CiteULike 
Connotea 
Del.icio.us What's this?
J Natl Cancer Inst 2007 99: 1534-1543.
J Natl Cancer Inst 2008 100: 284-285.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||