MGMT 8825M Week 7 Autonomy Brief Example

The Engine May Say Yes, Only a Clinician May Say No: An Autonomy Brief Dividing a Health Plan's Prior Authorization Engine Into Four Functions, Setting a Level of Automation for Each and Guarding the Denial Reviewer Against the Ironies of Automation

Student Name

Walden University

MGMT 8825M: Ethics of Artificial Intelligence and Autonomous Systems

Instructor Name

Month Day, Year

The Engine May Say Yes, Only a Clinician May Say No: An Autonomy Brief Dividing a Health Plan's Prior Authorization Engine Into Four Functions, Setting a Level of Automation for Each and Guarding the Denial Reviewer Against the Ironies of Automation

Purpose

Northgate Health Plan, a composite regional insurer covering about 600,000 members, is replacing the manual review of prior authorization requests for outpatient imaging, physical therapy and durable medical equipment with a software engine. The engine reads each request and the attached clinical notes, checks them against the plan's coverage criteria and recommends approval or denial. About 310,000 requests a year fall within scope. This brief sets how much of that work the engine may do on its own.

Four Functions, Four Levels

Parasuraman et al. (2000) proposed that automation be judged function by function rather than as a whole. A system may acquire information, analyze it, select a decision and carry that decision out, and each function can be automated to a different degree, on a scale from fully manual to fully automatic. The table applies their model to the engine.

Table 1

Engine Functions and Proposed Levels of Automation

Levels follow the logic of Parasuraman et al. (2000); the labels high, full and low summarize positions on their scale.

The Asymmetry

The central proposal is asymmetric. The engine may approve on its own, but only a clinician may deny. The reasoning rests on the different harms of the two errors. A wrongful approval costs the plan the price of a scan or a course of therapy. A wrongful denial delays or prevents care, falls on a member who may be ill and is hard to reverse quickly. A person affected by a denial is also owed reasons from someone who can be held responsible for them. Automating approvals still captures most of the engine's value: Northgate's historical data show that about 80 percent of requests in scope are approved.

The Irony to Guard Against

Bainbridge (1983) described the ironies of automation: designers automate the tasks they can, leaving people to monitor the system and step in when it fails, yet monitoring is a task people do poorly, and operators who rarely intervene lose the skill and attention to intervene well. Parasuraman and Riley (1997) described the related danger of misuse, in which people come to rely on automation beyond what its reliability justifies.

The risk for Northgate is a clinician reviewer who sees 60 denial recommendations a day, agrees with nearly all of them and begins signing without reading. A nominal reviewer of that kind would fail the brief's own test, since the member would receive a denial no person had actually decided. Four safeguards address this. Each recommendation must show the unmet criterion and the note text behind it, not only a conclusion. Reviewers must record a reason in their own words when they agree with a denial. Each week, a sample of recommendations that the engine has deliberately flipped from approve to deny is inserted without labels, and reviewers who sign them are retrained. And the share of recommendations each reviewer overturns is reported monthly; a reviewer who overturns almost none is a signal to investigate, not a sign of efficiency.

What the Brief Asks the Plan to Adopt

The plan should adopt the levels in the table, write the asymmetry into its utilization management policy so that a later cost review cannot quietly automate denials, and fund enough clinician time to review denial recommendations at a pace of no more than about 30 per reviewer per day. A review of overturn rates and of member appeals after six months will show whether the reviewers remain in real control.

References

Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775-779. https://doi.org/10.1016/0005-1098(83)90046-8

Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230-253. https://doi.org/10.1518/001872097778543886

Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 30(3), 286-297. https://doi.org/10.1109/3468.844354