PUBH 8410 Week 11 Evaluation Report Example

Three Arms, Three Verdicts, and an Aim That Outlives One of Them: A First-Year Evaluation Report on Ridgeline County's Three-Door Treatment Program for Pregnant Residents, Judged for Merit and for Worth

Student Name

Walden University

PUBH 8410: Fundamentals of Leadership, System Evaluation, and Interventions in Public Health

Instructor Name

Month Day, Year

Three Arms, Three Verdicts, and an Aim That Outlives One of Them: A First-Year Evaluation Report on Ridgeline County's Three-Door Treatment Program for Pregnant Residents, Judged for Merit and for Worth

Executive Summary

Ridgeline County, which does not exist, ran its Bridge pathway for a year in three arms. Every result is reported in words rather than numbers, since the county is imaginary. The report reaches three verdicts. The emergency department arm should be made permanent. Result routing should continue and expand to the county's private obstetric practices. The jail arm, as designed, should end under the stopping rule the steering group signed before launch, and the county board should replace it with a contract requirement that the jail's medical provider continue medication for opioid use disorder for every pregnant person in custody.

Evaluation Questions

The questions below are copied from the intervention design memo rather than rewritten, so the report judges the pathway against its original commitments.

The first asks whether each arm delivered a first dose, continuation or contact to the share of eligible patients set before launch. The second asks whether patients reached by the pathway attended clinic visits and remained in treatment at delivery. The third asks whether the pathway caused harm, measured by emergency department length of stay, child welfare reports from pathway sites and jail staff overtime. The fourth asks what share of any improvement can reasonably be credited to the pathway.

Methods

The indicators, their sources and every target were locked in writing before the first enrollment. Process data came from the pathway log, pharmacy records, the navigator's list and clinic scheduling. Staff at each site were interviewed at months 6 and 12, four roles per site. The stopping rule was applied at each review point as written. The attribution question was examined through contribution analysis, which builds a reasonable claim by testing each link of the program's theory and checking rival explanations (Mayne, 2012).

Findings by Component

Emergency Department Arm

The arm met its first-dose target at every review after the first. Clinicians reported that having the navigator complete referrals made the order workable on crowded shifts. Most patients given a first dose attended a clinic visit within seven days, consistent with trial evidence that starting buprenorphine in an emergency department keeps more people in treatment than a referral alone (D'Onofrio et al., 2015). Length of stay for pathway patients stayed within the harm threshold. The pathway never paused.

Result Routing

Most flagged results were contacted within three days from the second month onward. The navigator's list became the only place in the county where every known pregnant resident with opioid use disorder appeared, and prenatal clinics began asking for it. The cost was one additional position.

Jail Arm

The jail arm launched three months after the emergency arm, once the contract was amended. At its first review, continuation within 24 hours fell below the alteration threshold, because verifying a prescription after hours often took two or three days. At its second review the steering group altered the design, moving continuation to an early-custody clinic visit with a community prescriber. Performance improved but stayed below the stopping threshold at the second and third reviews. Under the signed rule, two consecutive reviews below that level end the arm. Interviews explained why: the contractor had no prescriber on site at night, and short stays meant many people were released before verification.

Harm and Balancing Measures

Child welfare reports from pathway sites did not rise above the prior year. Jail staff overtime rose during the altered design. No patient already started lost treatment during any pause, since transfers to Hollis Street Community Health Center were arranged as planned.

Contribution

The attribution memo concluded with moderate confidence that the pathway contributed to more residents with little or no prenatal care being in treatment at delivery. Shorter nursery stays were better explained by the hospital's new nursery care model, which a similar program showed could reduce stays and morphine use regardless of maternal treatment (Grossman et al., 2017).

Judgment

Scriven (1991) distinguished merit, how well something performs by its own standards, from worth, its value in a particular setting given what it costs. The distinction lets the report give different verdicts without contradiction.

Ending the jail arm does not end the county's obligation. Guidance recommends medication treatment throughout pregnancy (American College of Obstetricians and Gynecologists [ACOG], 2017), and forced withdrawal in custody raises serious equity and legal concerns (Ahlbach et al., 2020). The arm failed as a pilot run by a contractor without night prescribers. A permanent contract term with funded on-site prescribing changes the rule itself rather than testing a workaround.

The verdicts also reflect what the stopping rule did and did not do. It ended an arm that the steering group, by its own account, would otherwise have extended for another year in the hope that night verification would improve. It did not end the emergency arm after a weak first review, because that arm sat above its stopping threshold and recovered once the navigator took over referrals. The rule worked as intended: it separated an arm that needed adjustment from one that needed replacement.

Table 1

Merit and worth by arm

Judgments combine the indicator results with interview evidence.

Limitations

Numbers in one county over one year are small, and chance may explain part of every trend. One emergency department's records were incomplete for two months. The contribution claim rests on a pattern across prenatal care groups and holds at moderate confidence only. Interview participants were chosen by site managers and may have been more favorable than their colleagues.

Monitoring Plan

The county will keep tracking first doses, navigator contacts, treatment at delivery and the three balancing measures quarterly. The expanded routing arm will report its first private-practice results at six months. The new jail contract term will be reviewed against the same continuation indicator, with the original threshold, after its first year.

References

Ahlbach, C., Sufrin, C., & Shlafer, R. (2020). Care for incarcerated pregnant people with opioid use disorder: Equity and justice implications. Obstetrics & Gynecology, 136(3), 576-581. https://doi.org/10.1097/AOG.0000000000004002

American College of Obstetricians and Gynecologists. (2017). Committee opinion no. 711: Opioid use and opioid use disorder in pregnancy. Obstetrics & Gynecology, 130(2), e81-e94. https://doi.org/10.1097/AOG.0000000000002235

D'Onofrio, G., O'Connor, P. G., Pantalon, M. V., Chawarski, M. C., Busch, S. H., Owens, P. H., Bernstein, S. L., & Fiellin, D. A. (2015). Emergency department-initiated buprenorphine/naloxone treatment for opioid dependence: A randomized clinical trial. JAMA, 313(16), 1636-1644. https://doi.org/10.1001/jama.2015.3474

Grossman, M. R., Berkwitt, A. K., Osborn, R. R., Xu, Y., Esserman, D. A., Shapiro, E. D., & Bizzarro, M. J. (2017). An initiative to improve the quality of care of infants with neonatal abstinence syndrome. Pediatrics, 139(6), Article e20163360. https://doi.org/10.1542/peds.2016-3360

Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270-280. https://doi.org/10.1177/1356389012451663

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Sage.