PUBH 6333 Week 10 Evaluation Memo Example

Memorandum

To: Injury Prevention Program Manager, Pruett County Health Department

From: Health Communication Intern

Date: Month Day, Year

Re: How we will evaluate the first mowing season of the campaign

The Problem With Counting

At the end of the mowing season, we will know that dealers hung about 2,900 tags, the stores handed out about 1,800 cards and the radio spot aired 32 times. Those numbers show that the campaign was delivered. They do not show that any man read a tag, remembered it or understood what it asked. This memo separates four levels of effect, argues that understanding is the level this campaign must reach to succeed and describes how to measure it with the resources we have.

Four Levels

Table 1

Levels of Effect, Measures and Feasibility

Note. Prepared for this course.

Why Understanding Is the Test

Health campaigns that are well designed tend to produce small to moderate effects (Noar, 2006), and one review estimated that the average campaign changes behavior in the intervention community by about 5 percentage points (Snyder, 2007). With effects of that size, a county campaign could not show a change in injuries in one season; although emergency departments nationwide see thousands of mower injuries to young people each year (Vollman & Smith, 2006), serious ones to small children are, fortunately, too rare in a county of this size to count reliably. Behavior change measured by self-report is real but prone to men saying what they think interviewers want to hear. Understanding, by contrast, can be measured honestly and directly: can a man explain, in his own words, the three actions the campaign asks for? If he cannot, no change in behavior can be attributed to the campaign. If he can, the campaign has done the part that communication controls.

The Follow-Up Interview

In the last two weeks of June, trained interviewers will stand at both farm supply stores on four Saturday mornings and approach men who say they operate a riding mower and have young children around at least weekly. Each interview takes about five minutes and follows three steps:

1. Recall without prompting: "Have you seen or heard anything this spring about mowing and kids?"

2. Understanding: "What would you say it asks people to do?" Answers are coded for each of the three actions: no rides, children inside with another adult, look back before reversing.

3. Behavior: "This season, has a child ridden on your mower?" asked neutrally, with an explanation that answers are anonymous.

The target is 120 interviews. Interviewers will also ask where the man encountered the message, so the team can see which channel reached whom.

Defining Success

The campaign will count its first season a success if at least half of men interviewed can name two of the three actions without prompting. That threshold is set before the interviews so that it cannot be adjusted to fit results. A comparison group would strengthen the evaluation; the team will ask the extension office in a neighboring county, where no campaign ran, to conduct 40 interviews at its own farm store with the same questions.

Limits

The interviews measure understanding shortly after the campaign's busiest weeks, so they cannot show whether understanding lasts into the next season.

Intercept interviews at farm stores miss men who do not shop there and overrepresent those who do, the same men most exposed to the counter card. The comparison county differs in ways the team cannot control. Self-reported rides will likely be undercounted. The season report will list these weaknesses next to the findings.

What We Will Report

The season report will give delivery counts, then recall, understanding and self-reported behavior, by channel and by age group, compared with the neighboring county. It will make one recommendation for the next season based on which channel produced the most understanding per dollar.

References

Noar, S. M. (2006). A 10-year retrospective of research in health mass media campaigns: Where do we go from here? Journal of Health Communication, 11(1), 21-42. https://doi.org/10.1080/10810730500461059

Snyder, L. B. (2007). Health communication campaigns and their impact on behavior. Journal of Nutrition Education and Behavior, 39(2 Suppl.), S32-S40. https://doi.org/10.1016/j.jneb.2006.09.004

Vollman, D., & Smith, G. A. (2006). Epidemiology of lawn mower-related injuries to children in the United States, 1990-2004. Pediatrics, 118(2), e273-e278. https://doi.org/10.1542/peds.2006-0056