MMHA 6200 Week 10 Community Program Evaluation Plan Example

Evaluation Plan: Objectives, Baselines and a Comparison Area for the East-Side Remediation Program

Student Name

Walden University

MMHA 6200: Creating Healthy Communities

Instructor Name

Month Day, Year

Evaluation Plan: Objectives, Baselines and a Comparison Area for the East-Side Remediation Program

Introduction

This plan sets the objectives by which the home remediation program in the east-side rental corridor will be judged. Each objective begins from a figure established earlier in the planning: the corridor's asthma emergency visit rate from the disparity brief, the homes reached from the budget and the trigger checklist from the intervention design. The plan adds an objective on the gap between the corridor and the rest of the county, and names untreated neighboring blocks as a comparison area, so that any change can be attributed to the program with some confidence rather than to trends affecting the whole city. All figures are illustrative.

The Objectives

Table 1 sets out the objectives in four tiers.

Table 1

Program Objectives by Tier (Illustrative)

Why Four Tiers

The four tiers let the program see where it is failing if the outcome does not move. If homes are not being assessed, the problem is outreach or staffing. If homes are assessed but repairs are not completed, the problem is the coalition's repair process or landlord cooperation. If repairs are completed but triggers persist, the problem is the quality of repairs or the checklist. And if triggers are removed but emergency visits do not fall, the program may not be reaching enough of the children who generate visits, or the link between the corridor's housing and its asthma visits may be weaker than the needs assessment suggested. The Community Guide review found that home-based multi-trigger interventions reduced acute care visits among children (Crocker et al., 2011), which supports setting an outcome target, but the size of the effect in the corridor is uncertain.

The Distribution Objective

Kindig and Stoddart's definition of population health includes how outcomes are spread across a population (Kindig & Stoddart, 2003). The distribution objective tracks the rate difference between the corridor and the rest of the county, the measure the disparity brief recommended because it describes the absolute burden of excess visits. Following that brief and guidance on measuring disparities (Keppel et al., 2005), the rate ratio will sit beside it in every report, showing relative as well as absolute progress.

The Comparison Area

Emergency visits for childhood asthma vary from year to year for reasons unrelated to any program, such as respiratory virus seasons and air quality. A fall in the corridor's rate could reflect a mild year rather than the program. To address this, the evaluation names a comparison area: blocks to the west of the corridor with similar housing age, renter share and child poverty, which the program will not serve in its first two years. If the corridor's rate falls more than the comparison area's, the difference is more plausibly due to the program. The comparison area is not a randomized control group, and it may differ in ways the evaluation cannot measure, but it is much stronger evidence than a before-and-after comparison of the corridor alone. If the program proves effective, the comparison area becomes the first expansion site, which is both fair to its residents and a natural next test. The coalition will tell the comparison area's tenants' association and school about the plan from the start, so that residents understand why their blocks are waiting and when they can expect the program.

Who Reviews the Results

Results are reviewed at three levels. The coalition's working group, including the tenants' association and the community development corporation, reviews process and intermediate measures monthly and adjusts outreach, scheduling and repairs as needed. The hospital's community benefit committee reviews all measures twice a year, with outcome and distribution measures annually, and decides whether to continue funding. And residents see the results at a community meeting each year, held at the school family resource center where the program began, because a program meant to serve the corridor should report to it. Sharing results with residents, including disappointing ones, builds the trust the program depends on to reach families who distrust institutions.

Data and Timing

Emergency visit data come from the county's shared emergency department dataset, which covers all county hospitals and was used for the disparity brief's baseline. Using the same source and the same definition, children under 18 with a primary diagnosis of asthma, keeps the baseline and later results comparable. School absence comes from the district's published annual reports, which define chronic absence the same way each year and cover both corridor schools and schools serving the comparison area. Process and intermediate measures come from program records and will be reported to the coalition monthly. Outcome and distribution measures will be reported annually, since a single quarter's emergency visits in a corridor of 6,200 children are too few to show reliable change.

Defining Failure Before the Results

If, after two years, the corridor's asthma emergency visit rate has not fallen more than the comparison area's, the program will not have shown an effect on its main outcome, even if process and intermediate targets are met. In that case, the coalition should examine where the chain broke before deciding whether to continue, redesign or end the program. The evaluation is designed so that the program could be shown to fail, which is what gives a positive result credibility. A program that could only succeed would teach the hospital nothing.

References

Crocker, D. D., Kinyota, S., Dumitru, G. G., Ligon, C. B., Herman, E. J., Ferdinands, J. M., Hopkins, D. P., Lawrence, B. M., & Sipe, T. A. (2011). Effectiveness of home-based, multi-trigger, multicomponent interventions with an environmental focus for reducing asthma morbidity: A Community Guide systematic review. American Journal of Preventive Medicine, 41(2 Suppl. 1), S5-S32. https://doi.org/10.1016/j.amepre.2011.05.012

Keppel, K., Pamuk, E., Lynch, J., Carter-Pokras, O., Kim, I., Mays, V., Pearcy, J., Schoenbach, V., & Weissman, J. S. (2005). Methodological issues in measuring health disparities (Vital and Health Statistics, Series 2, No. 141). National Center for Health Statistics.

Kindig, D., & Stoddart, G. (2003). What is population health? American Journal of Public Health, 93(3), 380-383. https://doi.org/10.2105/AJPH.93.3.380