Journal of the National Cancer Institute Advance Access originally published online on April 29, 2008
JNCI Journal of the National Cancer Institute 2008 100(9):683-684; doi:10.1093/jnci/djn106
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© The Author 2008. Published by Oxford University Press.
CORRESPONDENCE |
Re: Projecting Individualized Absolute Invasive Breast Cancer Risk in African American Women
Correspondence to: Beverly Rockhill Levine, PhD, Department of Public Health Education, 437K HHP Bldg, University of North Carolina, Greensboro, Greensboro, NC 27403 (e-mail: bjlevine{at}uncg.edu).
In their "Discussion," Gail et al. (1) seem to be saying that a problem with a model's concordance statistic (or area under a receiver operating curve [AUC]) is that it is inherently retrospective, implying that one should therefore not place much stock in it. The concordance statistic from the Gail model is retrospective in the same sense that the Gail model's parameter estimates (which produce the estimated risks) are retrospective; that is, all of these measures derive from existing data on risk factors and outcomes. As is the case with the model's parameter estimates, if the model is applied to a sample or population similar in key respects to that from which it has been derived, we can reasonably expect that the AUC will be generalizable. The key issue about the AUC is not, then, that it is retrospective or at least any more retrospective than parameter estimates.
The authors next state that "... if a model includes age only and predicts that every woman in the age range 60-64 years has a 1.7% risk of breast cancer ... then the AUC will be 0.50. Some would mistakenly construe this to mean that the model does not perform any better than a coin flip in predicting who will or will not get breast cancer. In fact, if one predicted that none of these women would be diagnosed with invasive breast cancer in the next five years, one would predict correctly for 100 – 1.7 = 98.3% of the women." This latter statement is true, but one does not need to rely on such a statistical model as support for a prediction that no women will get breast cancer during a period of 5 years. One simply needs general population incidence rates to see that only a small minority of women will develop disease over a period of 5 years. In recent years, the Gail model has been promoted as a decision-making aid for women with respect to chemoprevention of breast cancer. If the goal in providing risk estimates is to help women with such dichotomous decision making about prophylaxis, or even with entrance (or not) into a clinical trial, one cannot imply that a model is doing a good job by giving everyone the same risk estimate. A model that gives each person the same risk estimate has zero discriminatory ability—no individual appears better off or worse off than anyone else—which is precisely what a concordance statistic of 0.5 means in this example. Along these lines, the observed concordance statistic for the Gail model in African American women aged 60–64 years, 0.507, is indicative of no discriminatory ability. This result is troubling given that the age group of 60–64 years is the 5-year age group, of those examined by the authors, with the highest incidence.
Gail et al. (1) imply a key question in their "Discussion": might it be better, overall, to tell all women in the general population that they will not get breast cancer in the next 5 years and to be wrong only a small percentage of the time? (I am excluding from consideration here women known to be at high genetic risk—the Gail model is not intended for such women anyway.) According to the provocative work of Tu et al. (2), it is possible to lose information (ie, to increase entropy) by administering a screening test and by then attempting to segregate individuals into two groups. In other words, one could make mistakes in a higher percentage of the total population, and mistakes of a more harmful nature, by attempting to segregate individuals into two groups (eg, high and low risk) rather than by not segregating in the first place.
On the basis of the work of Tu et al. (2), it is likely that there will be a loss of information, and a consequent high risk of error, when attempts are made to segregate individuals into high-risk and low-risk groups by use of models with low discriminatory accuracy. It would be helpful if the authors could speak more on the intended and unintended uses of their models with respect to such dichotomous segregation of women.
REFERENCES
1. Gail MH, Costantino JP, Pee D, et al. Projecting individualized absolute invasive breast cancer risk in African-American women. J Natl Cancer Inst (2007) 99(23):1782–1792.
2. Tu XM, Litvak E, Pagano M. Issues in human immunodeficiency virus (HIV) screening programs. Am J Epidemiol (1992) 136(2):244–255.
Response to this Correspondence
CiteULike 
Connotea 
Del.icio.us What's this?
J Natl Cancer Inst 2008 100: 684.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||