HLTH 8451 Week 10 RIA Peer Review Example

Careful on Costs, Thin on Benefits: An Independent Peer Review of the Composite Regulatory Impact Analysis for the Heat Refuge Rule, With Seven Revisions Rated by Effect

Student Name

Walden University

HLTH 8451: Public Policy Analysis

Instructor Name

Month Day, Year

Careful on Costs, Thin on Benefits: An Independent Peer Review of the Composite Regulatory Impact Analysis for the Heat Refuge Rule, With Seven Revisions Rated by Effect

Summary Assessment

This report reviews the composite regulatory impact analysis (RIA) for the heat refuge rule, which compares a refuge-space mandate and a performance standard against a no-rule baseline and recommends the performance standard. The review follows the conventions of regulatory peer review: an overall judgment, then comments keyed to the sections of the analysis in the order they appear, each with a concern, the evidence and a specific request. Peer review of influential scientific information used in regulation is meant to test whether the analysis supports its conclusions (Final Information Quality Bulletin for Peer Review, 70 Fed. Reg. 2664, 2005), and this review asks that question of each section.

The analysis is careful in its costs and weaker in its benefits. Equipment costs rest on quotes for three facility sizes, are annualized correctly and are tested across a plausible range. The benefit estimates, which drive the result, rest on a mortality effect transferred from a different population and event, and the analysis says less about that transfer than it should. The recommendation may well be right, but two of the revisions requested below could change it.

Comments on the Baseline

The baseline is clearly specified and removes facilities already compliant, those under strict state rules and those expected to adopt voluntarily. The voluntary adoption share of 8 percent, however, appears to be assumed rather than estimated. No data source or method is given for it, and the alternative baseline sets it to zero rather than testing a higher value. Since an error in this share moves costs and benefits together, its effect on net benefits is moderate, but the analysis should either estimate the share from evidence, such as adoption rates after past regional outages, or describe it as an assumption and test a range.

Comments on the Benefits

This section receives most attention because it is where the analysis is weakest. The central mortality effect is the odds ratio of 1.12 for death within 30 days after power loss in Florida nursing homes following Hurricane Irma (Skarha et al., 2021). Using that estimate for heat emergencies elsewhere is a benefit transfer: a value estimated in one setting is applied in another. Benefit transfer is most defensible when the study and policy settings are similar in population, exposure and outcome (Johnston et al., 2015). Here they differ in two ways.

First, the exposure differs. The Irma estimate captures everything that followed power loss after a hurricane, including disrupted staffing, supply problems and the stress of the storm, not heat alone. The RIA then scales the effect across regions using temperature and mortality relationships estimated in whole-city populations (Gasparrini et al., 2015). Those relationships describe community-dwelling people, most of them younger and healthier than nursing home residents, and they relate outdoor temperature rather than indoor temperature to deaths. Applying them to frail residents indoors may understate the effect of heat on the most vulnerable or overstate the effect of an outdoor reading on an indoor environment; the direction of bias is not clear.

Second, the population differs only partly. The Irma cohort is a nursing home population, which is a strength. But it is from one state, with a high share of facilities already accustomed to heat. The RIA should discuss whether residents in northern states, where buildings are less often cooled and residents less acclimatized, face a larger effect. The Ontario evidence that extreme heat raised mortality in homes without air conditioning (Katz et al., 2026) could serve as a second source for that comparison.

The request is that the RIA present the benefit transfer explicitly, report results using the Ontario estimate as an alternative, and discuss the likely direction of bias from each source. As the uncertainty section itself shows, the mortality effect has the widest influence on net benefits of any parameter.

Comments on the Costs

The capital cost estimates are well supported and need no revision. Energy costs are less complete. The RIA values fuel and testing at average prices, but heat emergencies are the periods when electricity is most expensive and when facilities would run generators longest. The analysis should estimate energy costs under peak pricing and extended runtime, at least as a sensitivity case.

Comments on Small Entities

The Regulatory Flexibility Act requires agencies to consider the effect of rules on small entities and, where the effect is significant for a substantial number of them, to analyze alternatives that reduce the burden (5 U.S.C. §§ 601-612). Most nursing homes would qualify as small entities. The RIA reports costs by facility size but does not analyze rural facilities separately, even though its own government failure discussion identifies small rural homes as the most burdened. A separate estimate of cost per resident for small rural facilities, and of the number that might reduce beds or close, would meet the act's intent and inform the choice between alternatives.

Comments on Distribution

The RIA correctly treats higher Medicaid payments as transfers rather than costs. It does not say who bears them. If states raise rates to cover compliance, state budgets and federal matching funds absorb the cost; if they do not, facilities with many Medicaid residents absorb it, and those facilities already receive payment below cost (Bowblis et al., 2026). The analysis should state its assumption about pass-through and show the distributional effect under each case.

Comments on Uncertainty

The uncertainty analysis is thorough and its tests were specified in advance, which is a strength. Its Monte Carlo simulation should report results with the alternative mortality source as a separate run, so that structural uncertainty about the benefit transfer is not hidden inside a single distribution.

Recommended Revisions

Table 1

Seven Recommended Revisions and Their Likely Effect on the Conclusion

Ratings refer to the likely effect on which alternative is recommended, not on the size of the estimates.

Revisions 2 and 3 should be completed before the analysis is finalized. The others would strengthen it and could be completed in the final draft.

References

Bowblis, J. R., Miller, E. A., Simpson, E., Karon, S., & Cohen, M. A. (2026). Does Medicaid cover the cost of nursing home care? Variation by ownership status, payer-mix, and staffing level. Medical Care, 64(1), 29-37. https://doi.org/10.1097/MLR.0000000000002252

Final information quality bulletin for peer review, 70 Fed. Reg. 2664 (January 14, 2005).

Gasparrini, A., Guo, Y., Hashizume, M., Lavigne, E., Zanobetti, A., Schwartz, J., Tobias, A., Tong, S., Rocklöv, J., Forsberg, B., Leone, M., De Sario, M., Bell, M. L., Guo, Y.-L. L., Wu, C.-F., Kan, H., Yi, S.-M., de Sousa Zanotti Stagliorio Coelho, M., Saldiva, P. H. N., . . . Armstrong, B. (2015). Mortality risk attributable to high and low ambient temperature: A multicountry observational study. The Lancet, 386(9991), 369-375. https://doi.org/10.1016/S0140-6736(14)62114-0

Johnston, R. J., Rolfe, J., Rosenberger, R. S., & Brouwer, R. (Eds.). (2015). Benefit transfer of environmental and resource values: A guide for researchers and practitioners. Springer.

Katz, G. M., Brown, K. A., Giannakeas, V., & Stall, N. M. (2026). Air conditioning in nursing homes and mortality during extreme heat. JAMA Internal Medicine, 186(2), 243-251. https://doi.org/10.1001/jamainternmed.2025.6595

Regulatory Flexibility Act, 5 U.S.C. §§ 601-612 (2018).

Skarha, J., Gordon, L., Sakib, N., June, J., Jester, D. J., Peterson, L. J., Andel, R., & Dosa, D. M. (2021). Association of power outage with mortality and hospitalizations among Florida nursing home residents after Hurricane Irma. JAMA Health Forum, 2(11), e213900. https://doi.org/10.1001/jamahealthforum.2021.3900