HLTH 8035 Week 9 Screening Evaluation Study Critique Example

Study Critique: Longer Survival Among Screen-Detected Colorectal Cancers in a County Outreach Evaluation, and How Lead Time, Length Bias and Overdiagnosis Could Produce the Gap Without Saving a Life

Student Name

Walden University

HLTH 8035: Epidemiology: Decoding the Science of Public Health

Instructor Name

Month Day, Year

Study Critique: Longer Survival Among Screen-Detected Colorectal Cancers in a County Outreach Evaluation, and How Lead Time, Length Bias and Overdiagnosis Could Produce the Gap Without Saving a Life

The Evaluation as Its Authors Framed It

The composite evaluation describes a county outreach program that mailed stool-based screening kits to residents aged 50 to 75 who were overdue for colorectal cancer screening. Over five years, the program identified 112 cancers through positive kits followed by colonoscopy. The evaluators compared these screen-detected cancers with 340 colorectal cancers diagnosed in the county over the same period after people developed symptoms. Five-year survival was 81% among screen-detected cases and 58% among symptom-detected cases, and the report concluded that the outreach program "substantially improved survival" and should be expanded.

The Design

This is a comparison of self-selected groups within a non-randomized intervention. People did not choose their cancers' route to diagnosis at random: those who returned kits differ from those who did not, and the cancers found by screening differ from those that announce themselves through symptoms. More fundamentally, the outcome, survival measured from diagnosis, starts its clock at different points in the natural history of disease for the two groups. That choice of outcome is where the evaluation breaks, because survival from diagnosis can rise even when nobody lives a day longer (Welch et al., 2000).

Lead Time

Screening finds cancers earlier than symptoms would. If a person's cancer is detected by a kit three years before it would have caused symptoms, and the person dies at the same age either way, their survival from diagnosis is three years longer simply because the clock started earlier. Nothing about the date of death changed. In the composite evaluation, any lead time adds directly to the screen-detected group's survival, so a gap of 23 percentage points at five years could arise even if the program prevented no deaths. Lead time is built into any comparison of survival between screened and unscreened cases, which is why survival from diagnosis cannot by itself show that screening works.

Length-Biased Sampling

Cancers grow at different speeds. A slow-growing cancer spends a long time in the detectable but symptom-free phase, so a periodic test has many chances to find it. A fast-growing cancer may move from undetectable to symptomatic between screening rounds and be diagnosed after symptoms. Screening therefore samples disproportionately from slow-growing tumors, which carry a better prognosis regardless of when they are found. The screen-detected group in the evaluation is enriched for indolent disease and the symptom-detected group for aggressive disease, so their survival would differ even if detection timing had no effect at all.

Overdiagnosis

Screening also picks up tumors that, left alone, would never have troubled their owner before death from something else. When such cancers are diagnosed, the people who have them are counted as cancer cases and almost all survive, inflating the screened group's survival while no life is extended (Welch & Black, 2010). For colorectal cancer, overdiagnosis is thought to be smaller than for some other cancers, because screening often finds and removes precursor polyps, but it is not zero, and in a five-year evaluation it would push in the same direction as the other two biases.

A Domain-by-Domain Judgment Against a Target Trial

The ROBINS-I tool judges a non-randomized study by comparing it with the randomized trial it is trying to emulate, assessing bias across domains from confounding to selection to measurement of outcomes (Sterne et al., 2016). The target trial here would randomize overdue residents to receive a mailed kit or usual care and compare colorectal cancer deaths per head of population in each arm, an approach to specifying what an observational analysis should mimic (Hernán & Robins, 2016). Against that target, the evaluation has serious risk of bias from confounding, since kit returners differ from non-returners in health and health behavior; critical risk from selection into the comparison, since the groups are defined by how cancers were found rather than by assignment; and critical risk in the measurement of outcomes, since survival from diagnosis is a biased measure for this question. The overall judgment is critical risk of bias: the evaluation cannot answer whether the program reduces deaths.

The Outcome That Would Answer the Question

The question is whether outreach reduces deaths from colorectal cancer in the county's population. The outcome that answers it is population colorectal cancer mortality, deaths per 100,000 residents, compared between people offered outreach and people not offered it, ideally by randomizing invitations or rolling out the program in stages across areas. Mortality counted per head of population is immune to lead time, length bias and overdiagnosis, because it does not depend on when or whether anyone is diagnosed. Stage at diagnosis and the incidence of advanced cancer could serve as interim measures. Five-year survival among cases should be reported, if at all, with a statement that it cannot show benefit.

References

Hernán, M. A., & Robins, J. M. (2016). Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology, 183(8), 758-764. https://doi.org/10.1093/aje/kwv254

Sterne, J. A. C., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., Carpenter, J. R., Chan, A.-W., Churchill, R., Deeks, J. J., Hróbjartsson, A., Kirkham, J., Jüni, P., Loke, Y. K., Pigott, T. D., . . . Higgins, J. P. T. (2016). ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 355, Article i4919. https://doi.org/10.1136/bmj.i4919

Welch, H. G., & Black, W. C. (2010). Overdiagnosis in cancer. Journal of the National Cancer Institute, 102(9), 605-613. https://doi.org/10.1093/jnci/djq099

Welch, H. G., Schwartz, L. M., & Woloshin, S. (2000). Are increasing 5-year survival rates evidence of success against cancer? JAMA, 283(22), 2975-2978. https://doi.org/10.1001/jama.283.22.2975