| Course | PHLT 8044 Advanced Program Evaluation |
|---|---|
| Week | Week 9 |
| Paper type | Analysis plan |
| Length | About 1,000 words, 6 pages |
| Format | APA 7 student paper |
| School | Walden University |
| Program | PhD in Public Health |
| Updated | October 2026 |
Free sample paper for PHLT 8044 Week 9
Analysis Plan for the School Water Access Evaluation, Fixed Before Any Outcome Is Known: The Primary Comparison, the Decision Rule and the Result That Will Be Read as Failure
Student Name
Walden University
PHLT 8044: Advanced Program Evaluation
Instructor Name
Month Day, Year
Analysis Plan for the School Water Access Evaluation, Fixed Before Any Outcome Is Known: The Primary Comparison, the Decision Rule and the Result That Will Be Read as Failure
Why Fix the Plan Now
An analysis plan written after the data arrive can always find something that looks like success: a subgroup, a different outcome, a different time window. This plan closes that door by fixing the primary comparison, the model, the decision rule and the result that will count as failure while every post-installation outcome is still unknown. It will be date-stamped and deposited in a public registry before the first post-installation sales extract is opened, a practice that separates confirmatory from exploratory analysis (Nosek et al., 2018). Anything not specified here will be labeled exploratory when reported, however interesting it turns out to be.
Primary Comparison
The primary outcome is the weekly count of sugary drink servings sold in each cafeteria, divided by enrollment and taken from sales records for each of the 70 schools: 30 receiving stations this fall and 40 on the waiting list. The estimate is a difference in differences: the change in the outcome in program schools from the two baseline school years to the evaluation year, minus the same change in waiting-list schools. It will be estimated with a regression of school-week outcomes on an indicator for program schools in the evaluation year, with fixed effects for each school and for each calendar week, so that stable differences between schools and seasonal patterns shared by all schools are removed (Wing et al., 2018). Standard errors will be clustered at the school level, since weekly observations from the same school are not independent; treating them as independent would overstate precision, a problem documented in difference-in-differences studies with serially correlated outcomes (Bertrand et al., 2004). With 70 schools, clustered errors should perform adequately, and a wild cluster bootstrap will be reported as a check.
The effect will be expressed both in servings per student per week and as a percentage of the program schools' baseline mean.
Checking the Key Assumption First
Before the primary estimate is computed, the analyst will estimate an event-study version of the model using the two baseline years, testing whether the two groups' purchases were already moving together before installation. If the pre-period coefficients jointly differ from zero at the 0.10 level, or show a visible divergence, the primary estimate will still be reported but labeled as resting on a questionable assumption, and a sensitivity analysis adjusting for group-specific trends will be presented alongside it.
The Decision Rule
The threshold reflects a judgment made in advance with the intended users about what size of effect would matter for their decision, not a statistical convention. A statistically significant 2% reduction would be reported honestly as small.
Table 1
Pre-Specified Interpretation of the Primary Estimate
| Result | Condition | Reading | |
|---|---|---|
| Success | Reduction of at least 10% of the baseline mean, with a 95% confidence interval excluding zero | The program reduced cafeteria sugary drink purchases by a meaningful amount | |
| Inconclusive | Confidence interval excludes zero but the reduction is under 10%, or the point estimate is at least 10% but the interval includes zero | The program may have an effect, but the evidence does not establish a meaningful one | |
| Failure | Point estimate below a 10% reduction with the confidence interval including zero, or an increase whose interval excludes zero | First-year cafeteria purchases of sugary drinks did not fall measurably | |
Note. The 10% threshold was agreed with the program manager and budget office as the smallest reduction that would justify expansion at the program's cost.
Secondary Analyses
Three secondary analyses are specified now. First, whole-day sugary drink intake from the previous-day recall in 24 schools will be analyzed as a difference in differences of school means between baseline and follow-up, to test whether school purchase changes are offset outside school. Second, the primary model will be estimated separately for schools above and below the median rate of meal-subsidy eligibility, to examine whether effects differ by school income. Third, the primary outcome will be recomputed including flavored milk as a sugary drink. These analyses inform interpretation but will not override the decision rule.
Power and Precision
The design's precision should be stated before results exist, so that a null finding can be interpreted honestly. Using the two baseline years, the spread in year-to-year change in purchases across schools suggests a standard error for the primary estimate of about 0.11 servings per student per week with 30 program and 40 waiting-list schools. That gives roughly 80% power to detect a reduction of about 0.30 servings, or 12.7% of the baseline mean, and only about 60% power to detect a reduction of exactly 10%. The evaluation is therefore well equipped to detect an effect somewhat larger than the decision threshold and less well equipped to confirm one right at it. An inconclusive result near 10% must be interpreted with that limit stated, and the report will state it.
Missing Data and Exclusions
Weeks when a school was closed, or when sales data are missing because of system outages, will be excluded rather than imputed, and the number excluded will be reported by group. A school whose stations were not installed by the end of October will be analyzed in a separate sensitivity analysis and excluded from the primary estimate, as specified in the data collection plan. No school will be excluded on the basis of its outcomes.
What Will Not Be Done
The analysis will not test additional outcomes as primary, will not search for subgroups beyond the one specified and will not change the comparison period. Shadish et al. (2002) note that flexibility in analysis inflates the chance of finding an effect that is not there; these restrictions keep the reported result tied to the question asked.
Reporting
Results will be reported in the order specified here: the pre-trend check, the primary estimate with its interval and its reading under the decision rule, then secondary analyses, then any exploratory findings, labeled as such.
References
Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in-differences estimates? The Quarterly Journal of Economics, 119(1), 249-275. https://doi.org/10.1162/003355304772839588
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018). Designing difference in difference studies: Best practices for public health policy research. Annual Review of Public Health, 39, 453-469. https://doi.org/10.1146/annurev-publhealth-040617-013507
The Walden PHLT 8044 Week 9 instructions
The ninth assignment usually asks for an analysis plan written before outcome data are available. Faculty expect the primary comparison and model specified precisely, including how clustering and seasonality are handled, the check of the design's key assumption, and a decision rule that defines success, failure and an inconclusive result in advance. The smallest effect worth acting on should be agreed with intended users rather than set by convention. Secondary analyses should be listed and limited, and rules for missing data and exclusions stated. Strong plans also report the design's statistical power, so a null result can be interpreted honestly, and list analyses that will not be done. Registration with a date stamp before data are opened is increasingly expected. Three to four pages is typical.
How the PHLT 8044 Week 9 example is put together
An opening section explains why the plan is fixed now and commits to registration. The primary comparison section defines the outcome, the regression with school and week fixed effects, clustered standard errors and a bootstrap check. A section describes the event-study test of pre-period trends and what happens if it fails. A table defines success, inconclusive and failure results, with a note on how the 10% threshold was agreed. Secondary analyses cover whole-day recall, school income and flavored milk. A power section estimates the standard error from baseline data and states the detectable effect. Missing data and exclusion rules, a list of analyses that will not be done and a reporting order close the plan.
Reading the PHLT 8044 Week 9 rubric
The plan is graded on precision and on commitments made before results exist. Vague promises to analyze the data appropriately earn little. Credit goes to a fully specified model, a stated approach to clustered and seasonal data, a pre-trend check with consequences, and a decision rule with numerical thresholds tied to users' needs. Honest power analysis that admits limited sensitivity at the threshold is a mark of strong work. Secondary analyses should be few and clearly subordinate. Rules for missing data should avoid exclusion based on outcomes. A list of analyses that will not be done and a fixed reporting order show understanding of how flexibility inflates false findings. Accurate citation of difference-in-differences and preregistration literature supports the plan.
PHLT 8044 Week 9 help from the desk
Analysis plans fall short when they leave room to choose the analysis after seeing results. Specify the model, the outcome, the comparison period and the handling of clustering. Decide with your users what effect size would matter and write down what will count as success and failure. Check the design's key assumption first. Report how much power the design has. Limit secondary analyses and say what you will not do. Plans for any outcome evaluation gain credibility from this level of advance commitment.
Get a PHLT 8044 Week 9 example written to your instructions
Share the Week 9 prompt and rubric with your design memo and measurement plan, and the analysis is specified to fit them. A first plan costs nothing and arrives inside 24-48h. Threshold values that belong to your funder or program are left as labeled slots, since only they can say what difference matters.
More PHLT 8044 papers and PhD in Public Health samples
- PHLT 8044 Week 1: Monitoring Versus Evaluation Post
- PHLT 8044 Week 2: Water Program Evaluability Study
- PHLT 8044 Week 3: Water Access Program Logic Model
- PHLT 8044 Week 4: Water Program Evaluation Questions
- PHLT 8044 Week 5: Selection Bias Peer Reply
- PHLT 8044 Week 6: Difference-in-Differences Memo
- PHLT 8044 Week 7: Indicators and Credibility Plan
- PHLT 8044 Week 8: Data Collection Schedule Plan
- PHLT 8044 Week 10: First-Year Findings Memo
- PHLT 8044 Week 11: Second-Year Evaluation Plan
- PHLT 8042 Week 9: Care Home Visiting Limits Memo
- PHLT 8040 Week 9: Septic Coalition Strategy Memo
- PHLT 8034 Week 9: Bottling Line Noise Control Brief
- PHLT 8032 Week 9: APA Results Tables
PHLT 8044 Week 9 questions, answered
Who decides what counts as excellent or poor?
Those who will use the evaluation to decide, working with the evaluator, before results exist. In the sample that is the state program director and the funder, and the rubric records their agreement. Davidson's approach makes those judgments explicit rather than leaving them to the analyst's discretion after the fact, which is exactly what this week guards against.
Why specify a smallest worthwhile difference?
Because any difference can be called meaningful after it appears. Fixing in advance the size that would justify continued funding lets the verdict follow from the data rather than from how the data are described. The number itself is left to the funder's own judgment; the sample shows where it is recorded and how it governs.
What if the primary analysis turns out to be inappropriate?
The plan allows a change only if it is recorded before outcome data are examined, with the reason stated and the original kept visible. A change made after seeing results is reported as exploratory. The sample includes that rule because a plan that can be revised silently offers none of the protection it exists to provide.
Where can I find a free PHLT 8044 Week 9 sample paper?
This page reproduces the complete Week 9 analysis plan, fixing the difference-in-differences model, a numerical decision rule, power and secondary analyses before any outcome data are opened.
Why report statistical power in an analysis plan?
So that a null or inconclusive result can be read honestly. This design has good power for effects somewhat larger than the 10% threshold but only modest power at the threshold itself.