Journal of the National Cancer Institute Advance Access originally published online on January 27, 2009
JNCI Journal of the National Cancer Institute 2009 101(3):213-214; doi:10.1093/jnci/djn463
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© The Author 2009. Published by Oxford University Press.
CORRESPONDENCE |
Response: Re: False-Positive Results in Cancer Epidemiology: A Plea for Epistemological Modesty
Affiliations of authors: Lifestyle, Environment and Cancer Group, International Agency for Research on Cancer, Lyon, France (PB); International Epidemiology Institute, Rockville, MD (JKM, RET, LL, WJB); Vanderbilt University Medical Center, Vanderbilt-Ingram Cancer Center, Nashville, TN (JKM, RET, LL, WJB); Laboratory of Epidemiology, Mario Negri Institute, Milan, Italy (CLV); Institute of Medical Statistics and Biometry, University of Milan, Italy (CLV)
Correspondence to: Paolo Boffetta, MD, PhD, Lifestyle, Environment and Cancer Group, International Agency for Research on Cancer, 150 cours Albert Thomas, 69008 Lyon, France (e-mail: boffetta{at}iarc.fr).
We are pleased to respond to the two letters to the Journal (1,2) concerning our commentary on false-positive findings in occupational and environmental cancer research. These letters only reinforce our call for scientific skepticism and epistemological modesty in the interpretation of epidemiological study results.
The commentary's first example of a likely false positive was the report of increased risk of breast cancer associated with high serum levels of the DDT [1,1,1,-trichloro-2,2-bis(p-chlorophenyl)ethane] metabolite DDE [1,1-dichloro-2,2-bis(p-chlorophenyl)ethylene]. Clapp and Kriebel (1) claim that lower blood levels of DDE over time account for the absence of increased risks of breast cancer associated with serum DDE levels in research studies subsequent to the initial publication of a positive effect, but this explanation is not supported by evidence. As shown in Table 1 of our commentary, virtually all of the blood samples used in the first four attempts to replicate (3–6) the initial positive study from New York (7) were collected before those used in the New York study, yet three of these studies showed a reduced risk of breast cancer among those with high DDE levels (4–6). Hence, although the conjecture proffered by Clapp and Kriebel (1) may be parsimonious, it is refuted by the epidemiological evidence.
With respect to the assertion of Clapp and Kriebel (1) that epidemiologists have "clear incentive[s]" to publish small, industry-sponsored negative studies, we presented a funnel plot showing results from studies evaluating the association between dioxin exposure and non–Hodgkin lymphoma, noting the conspicuous absence of small negative studies of this cancer among dioxin-exposed workers. Clapp and Kriebel (1) refer to the early publication of small negative studies of soft tissue sarcomas in relation to 2,3,7,8-TCDD [2,3,7,8 tetrachlorodibenzo-p-dioxin]exposure. Soft tissue sarcoma is a rare cancer; hence, even large cohorts will have few workers diagnosed with this malignancy, so a relative scarcity of published positive results is expected. Indeed, only four deaths from soft tissue sarcomas were observed across the multiple occupational cohorts in the National Institute for Occupational Safety and Health study (8) alluded to by Clapp and Kriebel (1). Furthermore, one of the studies showing an elevated rate of soft tissue sarcoma, which was published before the National Institute for Occupational Safety and Health review, was in fact an industry-conducted investigation (9).
Both Clapp and Kriebel (1) and Crosignani (2) seem to consider false-negative results to be more of a problem than false-positive results, although they provide no evidence from occupational epidemiology to support their beliefs. We of course recognize that false-negative results can and do occur, and there are examples from environmental epidemiology of early studies showing false-negative results. The important roles played by aflatoxins in liver cancer (10) and human papillomavirus in cervical cancer (11) (as well as other cancers) were identified only after several years of negative epidemiological investigations. The false-negative findings in these cases can be attributed largely to the low sensitivity of early methods of exposure assessment. Furthermore, as both examples show, negative early studies do not put an end to important biologically grounded lines of epidemiological research.
The issue of whether such examples exist in occupational and environmental cancer epidemiology, and therefore have impeded progress in identifying causes of cancer, should be assessed using scientific evidence rather than invoking an ideological litmus test regarding funding source. If a false-negative result exists, then a cumulative meta-analysis of the type we performed for acrylonitrile or DDE would show an increasing trend in the risk estimates over time. We are not aware of examples in which known occupational carcinogens were missed, or their detection delayed, because of early false-negative results. The authors of the letters (1,2) seem to presume that there are myriad carcinogens, especially of chemical origin, which are yet to be detected that imperil public well-being. Moreover, the letters represent a growing activist movement to accept any elevated risk ratio in environmental or occupational epidemiology as real and incontrovertible and to meet any criticism of such studies with accusations of deliberately creating uncertainty or doubt. This ominous trend will go a long way toward marginalizing epidemiology as a serious and credible science.
Crosignani (2) believes that false-positive results are not a major problem. He notes that attempts to replicate positive findings are common, in part because financial support is often easier to obtain for efforts to replicate reported positive associations and also because reports that confirm positive results are thought to be both more attractive to scientific journals and more likely to enhance a researcher's scientific reputation. These are precisely some of the common, nonscientific reasons that motivate investigators when they report positive epidemiological findings. These careerist incentives are major contributors to the bandwagon effect commonly seen in our field, which can lead to the apparent confirmation of false-positive findings.
We reject the claim that false-positive results are harmful only to commercial interests, and our commentary listed several adverse effects of false-positive findings on both the research community and the general population. We emphasized in our commentary several limitations inherent in epidemiological research that may lead to false-positive results. It may well be true that most epidemiologists are in some sense aware of these limitations, but it is clear from the literature that scientific skepticism and cautious interpretation are too seldom practiced. The fact that studies should be designed with the goal of minimizing the probability of false-negative, as well as false-positive, findings to every extent possible is not in debate.
To summarize, our commentary emphasized several limitations inherent in epidemiology research that may lead to false-positive results and overstatement of results. Although sources of error leading to false-negative results (eg, nondifferential misclassification of exposure or outcome or low statistical power, routinely invoked when risk ratios are >1 but not statistically significant) are generally well known and acknowledged, the thoughtful cautions required due to the problem of false-positive results are not sufficiently addressed in the practice of epidemiology. The issue is simple: the nonexperimental nature of our science necessitates the need for considerable skepticism and epistemic modesty, cognitive habits that are often missing in our field when reporting results and making claims.
REFERENCES
1. Clapp RW, Kriebel D. Re: False-positive results in cancer epidemiology: a plea for epistemological modesty. J Natl Cancer Inst. (2008) 101(3):211–212.
2. Crosignani P. Re: False-positive results in cancer epidemiology: a plea for epistemological modesty. J Natl Cancer Inst. (2008) 101(3):212–213.
3. Krieger N, Wolff M, Hiatt R, Rivera M, Vogelman J, Orentreich N. Breast cancer and serum organochlorines: a prospective study among White, Black, and Asian women. J Natl Cancer Inst. (1994) 86(8):589–599.
4. Høyer AP, Granjean P, Jørgensen T, Brock JW, Hartvig HB. Organochlorine exposure and risk of breast cancer. Lancet. (1998) 352(9143):1816–1820.[CrossRef][Web of Science][Medline]
5. Helzlsouer K, Alberg A, Huang HY, et al. Serum concentrations of organochlorine compounds and the subsequent development of breast cancer. Cancer Epidemiol Biomarkers Prev. (1999) 8(6):525–532.
6. Dorgan JF, Brock JW, Rothman N, et al. Serum organochlorine pesticides and PCBs and breast cancer risk: results from a prospective analysis (USA). Cancer Causes Control. (1999) 10(1):1–11.[CrossRef][Web of Science][Medline]
7. Wolff M, Toniolo P, Lee E, Rivera M, Dubin N. Blood levels of organochlorine residues and risk of breast cancer. J Natl Cancer Inst. (1993) 85(8):648–652.
8. Fingerhut MA, Halperin WE, Marlow DA, Piacitelli LA, et al. Cancer mortality in workers exposed to 2,3,7,8-tetrachlorodibenzo-p-dioxin. N Engl J Med. (1991) 324(4):212–218.[Abstract]
9. Bond GG, McLaren E, Lipps TE, Cook R. Update of mortality among chemical workers potentially exposed to chlorinated dioxins. J Occup Med. (1989) 31(2):121–123.[Web of Science][Medline]
10. International Agency for Research on Cancer. Some Traditional; Herbal Medicines, Some Mycotoxins, Naphthalene and Styrene. (2002) Lyon, France: IARC. 171–300. IARC Monographs on the Evaluation of Carcinogenic Risks to Humans, vol 82.
11. International Agency for Research on Cancer. Human Papillomaviruses. (2007) Lyon, France: IARC. 47–632. IARC Monographs on the Evaluation of Carcinogenic Risks to Humans, vol 90.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. K. Mclaughlin, C. La vecchia, R. E. Tarone, L. Lipworth, and W. J. Blot Response: Re: False-Positive Results in Cancer Epidemiology: A Plea for Epistemological Modesty J Natl Cancer Inst, January 20, 2010; 102(2): 134 - 135. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||