MBAX 6850 Week 8 Tutoring Program Evaluation Example

Does Tutoring Help Adults Read Better? A Program Evaluation of Lakeshore's One-to-One Tutoring for Its Board, Pairing Standardized Gains With Goals Learners Set and Keeping Activity Apart From Results

Student Name

Walden University

MBAX 6850: Introduction to the Nonprofit Sector

Instructor Name

Month Day, Year

Does Tutoring Help Adults Read Better? A Program Evaluation of Lakeshore's One-to-One Tutoring for Its Board, Pairing Standardized Gains With Goals Learners Set and Keeping Activity Apart From Results

The Board's Question

The board of Lakeshore, a made-up adult literacy group in northeast Ohio, asked a simple question at its spring retreat: does our one-to-one tutoring actually improve adults' reading? The board receives a count of learners served every quarter, but a count shows that the doors stayed open, not whether anyone reads better. This evaluation answers the board's question for last program year, using records Lakeshore already keeps. All figures are illustrative.

Methods in Plain Terms

Two measures are used, because each captures something the other misses. The first is a standardized reading assessment given at intake and again after at least 40 hours of tutoring, the same kind of test the state contract requires. It allows comparison across learners and with other programs. The second is the set of goals each learner names at intake, such as reading to a grandchild, passing the driver's license knowledge test or filling out a job application. Tutors record when a goal is reached, and a coordinator confirms it with the learner. This second measure reflects the mission's emphasis on goals adults set for themselves. Rossi et al. (2019) recommend that evaluations combine standardized outcome measures with measures tied to a program's own aims, so that neither alone defines success.

Outputs Apart From Outcomes

Outputs describe activity: how many people came and how much tutoring took place. Outcomes describe results: whether learners' reading improved and whether they reached their own goals. The two are kept apart because combining them, for example reporting "620 adults helped," would claim results for everyone who enrolled, including those who left after two sessions. Carman (2010) warns that nonprofits under pressure to show accountability often report outputs as if they were outcomes, which misleads funders and boards alike.

Table 1

Outputs and Outcomes, Last Program Year, Illustrative

Composite organization; figures illustrative.

What the Results Show

About half of retested learners gained at least one level on the standardized assessment, and about a third of all enrolled learners reached a goal they set. Gains were larger for learners who received more than 60 hours of tutoring, about 61 percent, than for those with 40 to 60 hours, about 44 percent. Goals reached most often were practical ones, such as reading mail or completing forms; goals tied to employment were reached less often within the year.

Results by Learner Group

Breaking the results down shows where tutoring works best. Learners who entered at the lowest reading levels were least likely to be retested, about 54 percent compared with 72 percent for those entering higher, because many found weekly sessions hard to sustain alongside work and family. Among those who were retested, however, the lowest-level learners made gains at about the same rate as others. Learners whose first language is not English made up about a third of tutoring enrollment and reached their own goals slightly more often than native English speakers, perhaps because their goals, such as a citizenship interview, were more specific. These patterns suggest that keeping the lowest-level learners engaged would raise overall results more than any change to the tutoring itself.

What the Results Cannot Show

Three limits matter. First, there is no comparison group, so the evaluation cannot show how much of the gain tutoring caused; some learners might have improved anyway through other classes or daily practice. Second, only two-thirds of enrolled learners were retested, and those who stayed long enough to be retested may have been the most motivated, so the 52 percent figure probably overstates the gain for all learners. Third, goal attainment is self-reported and confirmed by a coordinator, not independently verified. Weiss (1998) cautions that these limits are common in program evaluation and that stating them strengthens rather than weakens an evaluation's credibility.

Answer to the Board

The best available answer to the board's question is a qualified yes. Learners who stay in tutoring for at least 40 hours usually improve, and more hours are associated with larger gains. But one-third of learners leave before they can be retested, and the evaluation cannot say how much of the improvement is due to tutoring itself.

Recommendations

Lakeshore should focus on keeping learners engaged long enough to benefit, since retention appears to drive results, by matching learners and tutors more carefully and following up within a week when a learner misses a session. It should also report outputs and outcomes separately in all board and funder materials. If a funder offers support for a stronger evaluation, comparing tutored learners with similar adults on the waiting list would answer the causal question.

References

Carman, J. G. (2010). The accountability movement: What's wrong with this theory of change? Nonprofit and Voluntary Sector Quarterly, 39(2), 256-274. https://doi.org/10.1177/0899764008330622

Rossi, P. H., Lipsey, M. W., & Henry, G. T. (2019). Evaluation: A systematic approach (8th ed.). Sage.

Weiss, C. H. (1998). Evaluation: Methods for studying programs and policies (2nd ed.). Prentice Hall.