PUBH 8033 Week 8 Missing Data Memo Example

Memorandum

To: Director of Aging Services, Larkin County

From: Doctoral Student and Program Analyst

Date: Month Day, Year

Re: Missing follow-up data and what it means for the class's crash-per-mile estimate

What Is Missing

Crash involvement for all 730 applicants comes from the state crash file, linked by license number, so the crash outcome is nearly complete; only 25 applicants who moved out of state or died during the year lack a full year of records. But the class also planned to report crashes per 10,000 miles driven, and miles come from a twelve-month follow-up phone call. Of the 730 applicants, 161, or 22%, did not answer that call after three attempts: 18% of completers and 27% of waitlisted applicants. All figures are composite. This memo asks who those people were and how their silence affects the per-mile estimate.

Who Did Not Answer

Comparing responders and nonresponders on intake measures shows clear differences. Nonresponders were older (median 80 versus 76), more often reported a prior crash or near miss (38% versus 27%) and more often enrolled through the licensing office (31% versus 19%). Their crash rate in the following year, which we know from state records, was also higher: 7.5 crashes for every 100 nonresponders against 4.4 for every 100 who answered.

Why They Probably Drove Differently

Two opposite possibilities are plausible. Some nonresponders may have stopped or cut back driving after a health event or a crash, which would mean fewer miles than they reported at intake. Others may be harder to reach because they are still busy and driving a great deal. The intake data and higher crash rate suggest the first group is larger: older applicants with prior crashes are more likely to reduce driving. One trial of driver education for at-risk seniors recorded participants cutting back their driving exposure and avoiding challenging driving, which complicates any per-mile comparison (Owsley et al., 2004).

How the Estimate Shifts

Under every assumption, the per-mile rate ratio stays below 1, ranging from 0.68 to 0.83. The more waitlisted nonresponders cut their driving, the more favorable the class looks per mile. That is a warning, not good news: the class's apparent advantage per mile depends on assumptions about people we could not reach, and those people differ systematically between groups.

Table 1

Per-Mile Crash Rate Ratio Under Different Assumptions About Nonresponders' Miles

Note. Composite figures; estimates use state crash records for all applicants and imputed miles for nonresponders.

What This Means

Analyses that use only responders are safe only when missingness is unrelated to the outcome, the situation the missing data literature calls missing completely at random (Little & Rubin, 2019), and the intake comparison shows that condition does not hold here. The class's benefit per mile cannot be estimated with confidence from these data. The per-person estimate, which uses crash records for everyone, is more trustworthy, and after adjustment for who got seats it shows little or no effect. Random assignment to classroom safety teaching elsewhere did not measurably lower crashes (Owsley et al., 2004), and pooled reviews point the same way (Korner-Bitensky et al., 2009).

A Note on Those Lost to the Crash File

The 25 applicants who moved out of state or died during the year are a separate gap. Their crash records are incomplete, and some deaths may themselves have been crash-related. They are too few to change the estimate much, but the report should say how many there were and in which group.

Recommendation

Report the per-person estimate as the main result. Present the per-mile estimates only as a sensitivity analysis, with the assumptions stated. In future years, collect mileage at more than one point, using shorter text-message surveys, and record reasons for stopping driving so that missing exposure can be modeled rather than guessed.

References

Korner-Bitensky, N., Kua, A., von Zweck, C., & Van Benthem, K. (2009). Older driver retraining: An updated systematic review of evidence of effectiveness. Journal of Safety Research, 40(2), 105-111. https://doi.org/10.1016/j.jsr.2009.02.002

Little, R. J. A., & Rubin, D. B. (2019). Statistical analysis with missing data (3rd ed.). Wiley. https://doi.org/10.1002/9781119482260

Owsley, C., McGwin, G., Phillips, J. M., McNeal, S. F., & Stalvey, B. T. (2004). Impact of an educational program on the safety of high-risk, visually impaired, older drivers. American Journal of Preventive Medicine, 26(3), 222-229. https://doi.org/10.1016/j.amepre.2003.12.005