PHLT 8044 Week 10 First-Year Findings Memo Example

Memorandum

To: Director, State School Water Access Program

From: Lead Evaluator (DrPH student)

Date: Month Day, Year

Re: First-year results against the decision rule set in advance, and what they mean for next biennium's expansion

Bottom Line

Under the decision rule fixed before any outcome data were opened, the program meets the success criterion for its primary outcome: in its first year, sugary drink sales in participating cafeterias fell by about 13% more than in schools still waiting. Whether students offset that reduction outside school is not settled by the evidence. The decision on expansion is yours; this memo sets out what the evaluation can and cannot tell you.

Primary Result

Before computing the main estimate, we checked whether purchases in the two groups had moved in parallel during the two years before installation. They were: purchases moved in parallel, and the pre-period check showed no meaningful divergence.

Table 1

Primary and Secondary Results

Note. Composite results for illustration. Thirty program schools and 40 waiting-list schools; standard errors clustered by school.

What the Results Mean

The primary result is solid by the standard we agreed: a reduction of meaningful size whose interval excludes zero. It rests on objective sales data and survived the pre-trend check. Two qualifications matter for your decision. First, whether flavored milk is counted changes the answer at the margin; with milk included, the reduction falls just below the 10% threshold, because some students switched from soda to flavored milk rather than to water. Second, the whole-day recall points in the same direction as the sales data but is too imprecise to rule out the possibility that students made up some of the reduction after school. The logic model identified that link as the one most likely to break, and this evaluation cannot yet say whether it held.

The subgroup results, which were specified in advance, suggest the program does more in lower-income schools. That pattern fits the need data: daily sugary drink intake was higher at baseline in those schools, leaving more room for change.

Implementation

Stations were working and filters changed on schedule in 91% of station-months. Site visits found that in three program schools, stations had been placed far from the cafeteria because of plumbing constraints; those schools showed little change in purchases. This is an exploratory observation, not a planned analysis, but it agrees with the program theory that placement near the point of choice matters.

Cost

Total first-year cost for the 30 schools was $1.15 million, most of it installation. Dividing by the estimated servings displaced gives about $5.30 per serving displaced in the first year. Spreading installation over the stations' expected ten-year life, the ongoing cost falls to roughly $1.40 per serving displaced. Both figures depend on the primary estimate and inherit its uncertainty.

Options

Three courses are consistent with the evidence. Expansion to the 40 waiting-list schools as planned would extend a program that met its primary criterion. Expansion with a placement requirement, siting at least one station within 50 feet of each cafeteria entrance, would address the weakness seen in three schools. Prioritizing lower-income schools in the next wave would direct the program where its effect appears largest. A second year of follow-up in current schools, with a larger recall sample, would show whether the effect persists and whether students compensate outside school, which matters for any claim about health. Patton (2008) argues that findings serve decisions best when the evaluator lays out options and leaves the choice to the accountable official; that is the intent here.

A Note on Timing

The results come from a single school year in which the promotion ran in the fall and summer heat arrived early in May, both of which could affect purchases. Neither is likely to explain the difference between program and waiting-list schools, since both groups experienced the same calendar, but they are reasons to expect the second year's figures to differ somewhat.

Limits

The evaluation covers one year, measures school purchases rather than total intake with precision and does not address weight. Results come from schools that applied for the program, which may differ from schools that did not (Rossi et al., 2019). The full report documents every analysis in the registered plan, consistent with the accuracy and transparency expectations of the Program Evaluation Standards (Yarbrough et al., 2011).

References

Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). SAGE.

Rossi, P. H., Lipsey, M. W., & Henry, G. T. (2019). Evaluation: A systematic approach (8th ed.). SAGE.

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). SAGE.