NURS 8310 Week 5 Observational Study Critique Example

Fewer Emergency Visits After Housing, or Just Fewer Visits? Testing Four Alternative Explanations for an Observational Cohort Finding

Student Name

College of Nursing, Walden University

NURS 8310: Epidemiology and Population Health

Instructor Name

Month Day, Year

Fewer Emergency Visits After Housing, or Just Fewer Visits? Testing Four Alternative Explanations for an Observational Cohort Finding

The Finding Under Examination

The study examined here is a composite of a common type of published cohort, described in enough detail to critique. Using linked administrative data from one health system, investigators identified 420 adults who were placed in permanent supportive housing over two years and compared their emergency department visits in the twelve months before placement with the twelve months after. Visits fell from a mean of 6.2 per person in the year before placement to 3.1 in the year after. The authors conclude that housing placement reduced emergency department use by half.

The finding is plausible. Housing provides a place to store medications, rest, recover and be reached by clinicians, and it removes exposures to weather and violence that send people to emergency departments. This critique grants that plausibility and then asks whether four alternative explanations, selection, regression to the mean, confounding and differential measurement, could produce the same result even if housing did nothing at all.

Alternative 1: Selection

Selection concerns who ends up in the cohort. People placed in supportive housing are not a random sample of people experiencing homelessness. Placement programs often prioritize people with the highest measured needs, and people must be locatable, willing and able to complete paperwork and move in. Those who died, went to jail or disappeared from services before placement are not in the cohort. If the people who remained engaged long enough to be housed were also those whose health was stabilizing, the cohort would show a decline in visits for reasons unrelated to housing.

Could it produce the whole finding? Partly. Selection affects who is studied rather than the before-and-after change directly, but it shapes the kind of people whose trajectories are being observed.

Alternative 2: Regression to the Mean

Regression to the mean is the most serious threat here. Many placement programs use recent high use of emergency services as a criterion for priority. People selected because of an unusually high number of visits in a given year will, on average, have fewer visits the following year, simply because extreme values tend to be followed by values closer to the person's average (Barnett et al., 2005). The study's design, comparing the year before placement with the year after, is exactly the design most vulnerable to this effect.

Could it produce the whole finding? It could produce much of it. A 50% decline after selection on a peak year is well within what regression to the mean alone can produce in highly variable counts such as emergency visits.

Alternative 3: Confounding

Confounding occurs when a third factor affects both the exposure and the outcome. In a before-and-after design, the main confounders are other things that change around the time of placement. People who are placed in housing often receive new case management, are connected to primary care and may begin treatment for substance use or mental illness at the same time. Any of these could reduce emergency visits on its own. Changes in the health system itself, such as a new urgent care clinic or a change in emergency department policy, could also coincide with placement dates.

Could it produce the whole finding? Possibly part of it. The study cannot separate housing from the services that come with it, which may matter less for the program than for the causal claim.

Alternative 4: Differential Measurement

The study counted visits to the health system's own emergency departments. When people move into housing, they often move to a different neighborhood, and their nearest emergency department may belong to a different system. Visits at other hospitals would then disappear from the data after placement, producing an apparent decline. Research on readmissions has shown that a meaningful share of return hospitalizations occur at a different hospital than the original one (Kind et al., 2010), and the same pattern is likely for emergency visits by people who change address.

Could it produce the whole finding? It could produce a substantial part if many residents moved across system boundaries.

Putting the Threats Together

The four alternatives do not act alone, and some reinforce one another. Regression to the mean and selection both push the observed decline upward, because the people most likely to be placed are those whose use was temporarily at its peak and who stayed engaged. Differential measurement adds to the apparent decline by removing visits from view. Confounding by co-occurring services pushes in the same direction. None of the four works against the finding. That pattern matters: when every plausible bias points the same way, the observed effect is more likely to overstate the true one than to understate it.

A reader can also check what the study reports for clues. If the paper shows a steep rise in visits in the months just before placement, regression to the mean is likely. If it reports that many residents moved to a different part of the county, measurement loss is likely. If the comparison of before and after is the only analysis, none of the threats has been addressed. The composite study reports only the before-and-after means, so the reader cannot judge how much of the decline each threat explains.

What a Stronger Design Would Do

Each threat suggests a remedy. A comparison group of people who were eligible for housing but not yet placed, observed over the same calendar period, would address confounding by secular change and much of regression to the mean, since both groups would have been selected on the same high use. Following people from a point of eligibility, rather than from placement, would reduce selection by survival and engagement. Hernán and Robins (2016) describe how observational data can be analyzed to emulate a target trial, which here would mean defining eligibility, assignment and follow-up as a trial would, even without randomization. And linking to a regional or statewide database of emergency visits would address differential measurement.

Judgment

The finding that emergency visits fell after housing placement is real in the data, but the study cannot attribute the fall to housing. Regression to the mean alone could account for much of it, and measurement changes could add more. That does not mean housing does not reduce emergency visits; it means this design cannot tell. A health system deciding whether to fund housing should look for evidence from designs with concurrent comparison groups, and should use its own data to emulate such a comparison before claiming savings.

References

Barnett, A. G., van der Pols, J. C., & Dobson, A. J. (2005). Regression to the mean: What it is and how to deal with it. International Journal of Epidemiology, 34(1), 215-220. https://doi.org/10.1093/ije/dyh299

Hernán, M. A., & Robins, J. M. (2016). Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology, 183(8), 758-764. https://doi.org/10.1093/aje/kwv254

Kind, A. J. H., Bartels, C., Mell, M. W., Mullahy, J., & Smith, M. (2010). For-profit hospital status and rehospitalizations at different hospitals: An analysis of Medicare data. Annals of Internal Medicine, 153(11), 718-727. https://doi.org/10.7326/0003-4819-153-11-201012070-00005