HLTH 8475 Week 11 Utilization-Focused Report Example

Evaluation Report on the Rural Pharmacy PrEP Program After Twelve Months: Change, Not Continue or Stop, Written for the Board, the Medical Director and the Pharmacists Who Will Act On It

Student Name

Walden University

HLTH 8475: Advanced Program Implementation and Evaluation

Instructor Name

Month Day, Year

Evaluation Report on the Rural Pharmacy PrEP Program After Twelve Months: Change, Not Continue or Stop, Written for the Board, the Medical Director and the Pharmacists Who Will Act On It

The Recommendation

The district should change the rural pharmacy PrEP program, not continue it unchanged and not stop it. The program worked where it was delivered as designed and fell short where delivery broke down or where its design did not fit very small towns. The changes recommended below address both.

Who This Report Is For

Patton (2008) argued that evaluations should be designed for their intended users and the uses those users will make of them. This report's intended users are the district board, which will decide whether to fund a second year; the medical director, who supervises the collaborative practice agreement; and the pharmacists and pharmacy owners, who deliver the program. Each needs different things. The board needs a clear recommendation and its cost. The medical director needs the safety findings. The pharmacists need site-level results and specific changes.

How the Findings Were Produced

The evaluation used records the program kept for its own work and matched them against records kept by others. Pharmacy dispensing and service logs supplied persistence, adoption and fidelity. The contract laboratory supplied test results, which were matched to refills. The disease intervention specialists' case logs were matched to pharmacy intake records to measure referrals by source. Billing records showed sustainability. Pharmacists completed short acceptability and feasibility scales at launch and six months, and the coordinator interviewed pharmacists, two specialists and twelve clients. Every measure and its pre-set level had been fixed before the first client was seen, so the findings below are reported against standards the program chose in advance rather than against whatever turned out well.

What the Program Set Out to Do

The program offered HIV pre-exposure prophylaxis through four independent community pharmacies in a rural district, under a collaborative practice agreement, with referrals from walk-in clients, three primary care practices and the district's partner-notification team. It set its measures and pre-set levels before launch.

Findings

Safety

Not one client acquired HIV during the year while taking PrEP, and in 97 out of every 100 refills a negative HIV result was already on file, above the 95% level. The three exceptions occurred at one site during hospital lab delays, and all three tests were later negative.

Persistence

Of clients enrolled for six months or more, 58% had at least 80% of days covered at six months, just under the 60% level. Where a site had a closed counseling room, a second trained pharmacist and lab results that arrived on time, persistence reached 68% and 63%. At the other two, it was 47% and 42%.

Reach

The program started 118 clients in twelve months. Referrals from disease intervention specialists stopped in month three, after the specialist who championed them left, and were not restored until the process evaluation found the gap.

Adoption and sustainability

All four pharmacies adopted the program, and nearly all visits were billed or covered by the assistance pathway. Pharmacists at the two smaller sites reported unsustainable workload, and one left.

Unintended effects

Testing of clients increased diagnoses of other sexually transmitted infections, which were treated. Some clients in the smallest town traveled to a distant pharmacy for privacy.

What the Evidence Does Not Show

The program should not take credit for the district's drop in new diagnoses from 19 to 11. That fall is consistent with a return to the earlier average after a cluster year, the pattern Campbell (1969) warned about, and with other measures taken during the cluster response. The evaluation has no comparison group, so it cannot show that clients stayed on PrEP longer than they would have through other routes.

Why Change Rather Than Continue

Continuing unchanged would ask two pharmacies to keep delivering a program they could not deliver as designed. The persistence gap is concentrated where delivery failed, which suggests a type III problem, results judged at sites where the program was not fully in place (Dobson & Cook, 1980). Repeating that for another year would waste clients' time and the district's money.

Why Change Rather Than Stop

Stopping would discard a program that met its safety standard and its persistence target where delivered well, in a rural region where few other PrEP routes exist. A review of pharmacy PrEP services in states with restrictive prescribing authority found that most such services work through collaborative practice agreements, as this one does, and called for stronger evaluation rather than retreat (Wang et al., 2024).

Recommended Changes

1. Restore and protect referrals. Retrain disease intervention specialists, add pharmacy referral to their case log and name a backup champion.

2. Fix delivery at the weaker sites. Fund a private counseling space at pharmacy C and require two trained pharmacists per site before a site accepts new clients.

3. Address small-town privacy. Offer video or phone follow-up with a pharmacist in another town for clients who prefer it, so that clients in small towns need not discuss PrEP with a neighbor.

4. Reduce workload. Schedule PrEP visits in protected blocks rather than during dispensing peaks, and consider concentrating new starts at the two stronger sites if the smaller pharmacies cannot staff the program.

5. Improve measurement. Track referrals by source monthly and add a comparison with clients starting PrEP through primary care in neighboring districts.

The Objection a Skeptical Director Would Raise First

A county director's first question will be: if the program missed its persistence target and cannot show an effect on district diagnoses, why spend more on it? The answer is that the program met its target where it was delivered as designed, met its safety standard everywhere and is one of the few PrEP routes available in the district. The changes recommended cost less than starting over, and the evaluation has already identified what they need to fix. The report also commits to a decision point: if persistence at the strengthened sites does not reach 60% in the second year, the board should consider concentrating the program in fewer pharmacies or stopping it.

Limits

The evaluation covers one district, small numbers at each site and twelve months. Persistence is measured from dispensing records, which show whether medication was picked up, not whether it was taken.

References

Campbell, D. T. (1969). Reforms as experiments. American Psychologist, 24(4), 409-429. https://doi.org/10.1037/h0027982

Dobson, D., & Cook, T. J. (1980). Avoiding type III error in program evaluation. Evaluation and Program Planning, 3(4), 269-276. https://doi.org/10.1016/0149-7189(80)90042-7

Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Sage.

Wang, H., Guinn, D., Ramisetty, X. R., Giordano, T. P., & Poon, I. O. (2024). A review of studies on HIV pre-exposure prophylaxis in community pharmacies in states with restrictive pharmacist prescription authority in the United States. Pharmacy, 12(5), Article 144. https://doi.org/10.3390/pharmacy12050144