International Journal of
Radiation Oncology biology
COMMENTS We Need Better Figures! To the Editor: In the health-related scientific literature, the Kaplan-Meier method is commonly used to generate an estimate of survival when follow-up is less than complete (1). In a similar fashion, the cumulative incidence method is frequently invoked to estimate time to an event because it specifically adjusts for other competing risks (2). The results of these analyses are typically displayed in figures within scientific reports. These figures should be designed to provide the reader with sufficient information to make judgments about the content. It is my contention that the figures published in many reports do not meet this standard. Allow me to make my case. Although no rigorous guidelines have been published, one article published more than a decade ago suggested some best practices (3). In my opinion, there are at least four elements that should be provided for every Kaplan-Meier or cumulative incidence estimate in a scientific paper. First, a confidence interval should be included in the text for each estimate. In addition, three other elements should be evident from the figures: (1) the number of patients at risk; (2) censoring marks; and (3) the curves should be truncated when there are fewer than 10 patients at risk. In the November 1, 2013, edition of the International Journal of Radiation Oncology, Biology, Physics (the Red Journal), nine articles included figures with Kaplan-Meier or cumulative incidence methods. I examined each of the nine articles and determined whether the four elements listed above were present. The results are listed in Table 1. The good news is that censoring marks were evident in a majority of figures, but the remaining elements were rarely present, and I could find no evidence that any of the figures were truncated when fewer than 10 patients were at risk. The lack of confidence intervals is a recurring problem (4, 5). I freely admit that this analysis is not rigorous, and it could be that this one edition is not representative of the Red Journal, but I suspect a larger sample would produce the same results.
Analysis of figures Element
Confidence interval of estimate No. of patients at risk Censoring marks Curve truncated with <10 patients
Element present, n (%) 2/9 2/9 6/9 0/9
(22) (22) (66) (0)
Int J Radiation Oncol Biol Phys, Vol. 88, No. 1, pp. 236e242, 2014 0360-3016/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved.
One can argue that because no guidelines exist, it is presumptuous to require these elements. I take the position that these elements provide the reader with important information on the uncertainty (imprecision) associated with the estimates provided. With current statistical tools all of these elements are easily produced. There are multiple YouTube videos available that explain how. I encourage authors who intend to include KaplanMeier or cumulative incidence figures to include these elements and further encourage reviewers to demand them from the authors. W. Robert Lee, MD, MS, MEd Department of Radiation Oncology Duke University School of Medicine Durham, North Carolina http://dx.doi.org/10.1016/j.ijrobp.2013.10.027
References 1. Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc 1958;53:457-481. 2. Kalbfleisch JD, Prentice RL. The Statistical Analysis of Failure Time Data. New York: John Wiley & Sons, 1980. 3. Pocock SJ, Clayton TC, Altman DG. Survival plots of time-to-event outcomes in clinical trials: Good practice and pitfalls. Lancet 2002; 359:1686-1689. 4. Shakespeare TP, Holecek MJ. Should we be using confidence intervals when reporting results of oncology studies? Int J Radiat Oncol Biol Phys 1998;41:971-972. 5. Urbanic JJ, Lee WR. Confidence intervals and survival estimates: A systematic review of 3 oncology journals. Am J Clin Oncol 2006;29: 405-407.
In Reply to Lee As the Red Journal selection process has become increasingly rigorous in terms of its review of scientific content and overall priority, I agree that it is now time to turn our attention to the quality of the presentation. The editorial team is primed to pay additional scrutiny to the figures along the lines suggested by Dr Lee. In the submission portal, we will now be explicit about our requirement for: numbers at risk; censor marks; confidence intervals; and the appropriate truncation of Kaplan-Meier curves. We thank Dr Lee for helping us to raise the bar. Anthony L. Zietman, MD Editor-in-Chief