HLTH 6740 Week 9 Crisis Program Evaluation Example

Judging the Federal Crisis Counseling Program Against Its Own Claims: What the Cross-Site Evaluation Shows and What It Cannot

Student Name

Walden University

HLTH 6740: Disaster, Crisis, and Trauma

Instructor Name

Month Day, Year

Judging the Federal Crisis Counseling Program Against Its Own Claims: What the Cross-Site Evaluation Shows and What It Cannot

The Program

The Crisis Counseling Assistance and Training Program provides short-term, community-based support to disaster survivors after presidentially declared disasters. Federal law authorizes the President to provide professional counseling services and training for disaster workers to relieve mental health problems caused or aggravated by a major disaster (42 U.S.C. § 5183). In practice, states receive federal funding to hire and train crisis counselors, often paraprofessionals from the affected community, who go door to door, staff recovery centers and hold group sessions. Counselors provide emotional support, education about common reactions and links to other resources, and refer people with more serious problems to mental health treatment. This paper evaluates the program against the outcomes it claims, using what the program and the research literature have published about it.

What the Program Claims

The program's stated aims are to help survivors understand their reactions, reduce stress, develop coping strategies, connect with resources and return to their pre-disaster level of functioning, and to identify and refer people who need more intensive care. Those are the outcomes against which it should be judged. The program does not claim to treat disorders such as PTSD, which fits the evidence that most survivors recover without treatment while a minority need more (Bonanno et al., 2010).

What the Evidence Shows

The most substantial published evaluation examined programs in five states after Hurricanes Katrina, Rita and Wilma in 2005 (Norris et al., 2009). It drew on 2,850 participant surveys, 805 provider surveys and 132,733 encounter logs, aggregated to 50 counties, to test which program features were associated with better results. County-level ratings of participants' perceived benefits improved as service intensity increased, as services became more personal and less like brief contacts, as referrals for psychological care became more frequent and as provider job stress decreased. Counties with more providers holding advanced degrees achieved better results indirectly, by delivering more intensive services and making more referrals.

Measuring Against the Claims

Table 1 compares the program's claims with the evidence.

Table 1

Program Claims Compared With Published Evidence

What the Design Cannot Show

Three limits stand out. First, the main outcome was participants' perceived benefits, not change in symptoms or functioning. People may value support without it changing their course. Second, there was no comparison group of survivors who did not receive services, so improvements over time cannot be attributed to the program, especially since most survivors improve on their own. Third, the analysis was at the county level, so it shows which kinds of county programs performed better, not which individuals benefited. These limits do not mean the program fails; they mean the evidence cannot yet show whether it achieves its central claim, a return to pre-disaster functioning. A program can be valued, well run and still unproven, and an honest evaluation has to keep those three judgments apart.

How the Program Fits the Wider Evidence

The program's model fits the wider evidence better than some alternatives. Its emphasis on support, education, practical help and connection aligns with the five principles identified for immediate and mid-term intervention: safety, calming, efficacy, connectedness and hope (Hobfoll et al., 2007). It does not rely on single-session debriefing, which a systematic review found ineffective for preventing PTSD (Rose et al., 2002). And its referral function addresses the minority who need treatment. On design, therefore, the program is consistent with what research suggests; on outcomes, its effectiveness is not yet demonstrated.

What the Program Looked Like in the Composite County

When the April flood struck the invented county used across this course, the state's crisis counseling program hired eleven counselors from the community, most without clinical degrees, and supervised them through the county's behavioral health agency. In the first two months, counselors made several thousand brief contacts, mostly door to door and at the recovery center. The evaluation's findings suggest specific adjustments. Brief contacts were the easiest to count but the least associated with perceived benefit; the program could shift some effort toward repeat visits with households showing more distress. Referral rates were low, partly because counselors were unsure where to refer uninsured adults; a standing referral agreement with the community health center would raise them. And counselor stress was high by the second month, which the evaluation linked to weaker results; scheduled supervision and limits on caseloads would address it. None of these changes requires waiting for better outcome research.

What Would Strengthen the Evidence

Three changes would let future evaluations test the program's central claim. Programs could measure symptoms and functioning with brief standard scales at first contact and at follow-up, rather than only satisfaction. They could compare outcomes for people who received more and less intensive services, adjusting for initial severity, to see whether intensity predicts change as well as perceived benefit. And they could track referrals to see whether referred people received treatment. The 2005 evaluation already showed that larger data collection is possible after major disasters; adding outcome measures would make it far more informative.

Verdict

Judged against its own claims, the crisis counseling program is well designed and well received, and the published evaluation identifies features associated with better results: intensive and personal services, active referral and supported staff. Whether it helps survivors return to pre-disaster functioning more than they would on their own remains undemonstrated. A program manager should take from the evidence that intensity, referral and staff support matter, and should build outcome measurement into the next deployment.

References

Bonanno, G. A., Brewin, C. R., Kaniasty, K., & La Greca, A. M. (2010). Weighing the costs of disaster: Consequences, risks, and resilience in individuals, families, and communities. Psychological Science in the Public Interest, 11(1), 1-49. https://doi.org/10.1177/1529100610387086

Crisis counseling assistance and training, 42 U.S.C. § 5183 (2023).

Hobfoll, S. E., Watson, P., Bell, C. C., Bryant, R. A., Brymer, M. J., Friedman, M. J., Friedman, M., Gersons, B. P. R., de Jong, J. T. V. M., Layne, C. M., Maguen, S., Neria, Y., Norwood, A. E., Pynoos, R. S., Reissman, D., Ruzek, J. I., Shalev, A. Y., Solomon, Z., Steinberg, A. M., & Ursano, R. J. (2007). Five essential elements of immediate and mid-term mass trauma intervention: Empirical evidence. Psychiatry, 70(4), 283-315. https://doi.org/10.1521/psyc.2007.70.4.283

Norris, F. H., Hamblen, J. L., & Rosen, C. S. (2009). Service characteristics and counseling outcomes: Lessons from a cross-site evaluation of crisis counseling after Hurricanes Katrina, Rita and Wilma. Administration and Policy in Mental Health and Mental Health Services Research, 36(3), 176-185. https://doi.org/10.1007/s10488-009-0215-1

Rose, S., Bisson, J., Churchill, R., & Wessely, S. (2002). Psychological debriefing for preventing post traumatic stress disorder (PTSD). Cochrane Database of Systematic Reviews, 2002(2), Article CD000560. https://doi.org/10.1002/14651858.CD000560