HLTH 4205 Week 10 Research Article Appraisal Example

Appraising the Oregon Experiment's Clinical Outcomes Article: What Was Done, What Was Found and Where Readings of It Separate

Student Name

Walden University

HLTH 4205: Introduction to Research Methods and Analysis

Instructor Name

Month Day, Year

Appraising the Oregon Experiment's Clinical Outcomes Article: What Was Done, What Was Found and Where Readings of It Separate

The Article and Its Claim

This paper appraises the 2013 article reporting the effects of Medicaid coverage on clinical outcomes in the Oregon Health Insurance Experiment (Baicker et al., 2013). Its abstract concludes, in substance, that in the first two years Medicaid coverage produced no significant improvements in the measured physical health outcomes, while care use went up, more diabetes was detected and treated, lowered rates of depression and reduced financial strain. The appraisal treats that conclusion as a claim and checks it against what was done and found, section by section.

Question and Design

The study asked what effect Medicaid coverage has on the health and finances of low-income adults, a question that observational comparisons had struggled to answer because insured and uninsured people differ in so many ways. In 2008, Oregon drew names at random from a waiting list to decide who could apply for its adult Medicaid program. Because selection was random, people selected and not selected were similar at the outset, and later differences could be attributed to the chance of coverage. Not everyone selected enrolled, so the authors estimated the effect of coverage itself by scaling the difference between the groups by the increase in coverage that selection produced, an approach described in the study's first-year report (Finkelstein et al., 2012). For a causal question about coverage, this is about as strong a design as is possible outside a laboratory.

Participants and Measures

The clinical results come from in-person interviews and health examinations with about 12,000 adults in the Portland area, conducted roughly two years after the lottery. Measures included blood pressure, cholesterol, glycated hemoglobin as a marker of blood sugar control, a screen for depression, a measure of health-related quality of life, out-of-pocket medical spending and use of health services and medications. These are reasonable choices: they are objective where possible, relevant to common chronic conditions and comparable with other studies. The participants were people who had signed up for the lottery and lived near Portland, which limits generalization, as the earlier sampling note in this course described.

What Was Found

Coverage did not produce statistically significant changes in average blood pressure, cholesterol or glycated hemoglobin, or in the share of participants with elevated values. It did increase the share diagnosed with diabetes and the share using diabetes medication. It reduced the share screening positive for depression, and catastrophic medical bills paid out of pocket almost disappeared among the covered. Use of preventive care and prescription drugs increased (Baicker et al., 2013). Each of these findings is reported with a confidence interval in the article.

Where the Authors' Claim Matches the Results

The abstract's wording, no significant improvements in measured physical health outcomes, is accurate. It is limited to the measured outcomes, to the two-year period and to statistical significance. The positive findings on depression, diagnosis and finances are stated as findings, not as hints. On the whole, the authors' summary stays close to their results.

Where Readings Separate

The separation appears in how the physical health finding was read. Many commentators took no significant improvements to mean Medicaid has no effect on physical health. That does not follow. A result that is not statistically significant means the study could not distinguish the effect from zero; it does not show the effect is zero, a distinction that methodologists list among the most common misreadings of statistical tests (Greenland et al., 2016). Relatively few participants had elevated blood pressure, cholesterol or blood sugar at the start, so even a real benefit for those who did would be diluted across the whole sample, and the confidence intervals were wide enough to include improvements that clinicians would consider meaningful. Two years is also a short time for changes in cardiovascular risk to appear. A later review of the evidence across states made a similar point: coverage effects on physical health may be real but slow and concentrated among sicker people, which a study of this size and length could miss (Sommers et al., 2017).

The authors could have reduced this misreading by stating more plainly in the abstract what size of effect the study was able to rule out. That is the one place where the article's framing invited a conclusion its results do not support.

Strengths and Limits Summarized

The study's main strength is its random assignment, which makes its estimates of the effects of coverage far more credible than comparisons of insured and uninsured people. Its main limits are the size and composition of the sample for detecting physical health changes, the two-year horizon and the restriction to people who signed up for a lottery in one region. The financial and mental health findings are the most robust; the null physical health findings are the most often over-read.

Judgment

A careful reader can take three conclusions from the article: Medicaid coverage reduced financial strain and depression among these adults within two years; it increased the diagnosis and treatment of diabetes; and the study could not detect changes in average blood pressure, cholesterol or blood sugar over that period, which is different from showing that none occurred. The abstract supports the first two directly and supports the third when read with its confidence intervals. The article deserves its reputation as strong evidence, and its most famous finding deserves to be quoted with the qualification its own numbers require.

For a student of research methods, the article also shows how much depends on the step between a result and a sentence. The study's design and measurement were careful. What varied was the sentence each reader wrote about it. An appraisal that records the results first, then sets the authors' and others' sentences beside them, makes that gap visible and keeps the reader's own conclusion inside what the numbers support. Doing that for one strong article is good preparation for reading weaker ones, where the gap is usually wider and less often noticed.

References

Baicker, K., Taubman, S. L., Allen, H. L., Bernstein, M., Gruber, J. H., Newhouse, J. P., Schneider, E. C., Wright, B. J., Zaslavsky, A. M., & Finkelstein, A. N. (2013). The Oregon experiment: Effects of Medicaid on clinical outcomes. New England Journal of Medicine, 368(18), 1713-1722. https://doi.org/10.1056/NEJMsa1212321

Finkelstein, A., Taubman, S., Wright, B., Bernstein, M., Gruber, J., Newhouse, J. P., Allen, H., Baicker, K., & Oregon Health Study Group. (2012). The Oregon Health Insurance Experiment: Evidence from the first year. The Quarterly Journal of Economics, 127(3), 1057-1106. https://doi.org/10.1093/qje/qjs020

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337-350. https://doi.org/10.1007/s10654-016-0149-3

Sommers, B. D., Gawande, A. A., & Baicker, K. (2017). Health insurance coverage and health: What the recent evidence tells us. New England Journal of Medicine, 377(6), 586-593. https://doi.org/10.1056/NEJMsb1706645