MGMT 8825M Week 4 Accountability Memo Example

Memorandum

To: Board Safety Committee, a firm developing automated driving systems

From: Ethics and Governance Analyst

Date: September 14, 2026

Re: Where responsibility lay in the Tempe crash, and what it means for our test program

Conclusion

The safety operator was the last person able to prevent the March 2018 crash in Tempe, Arizona, but the developer that designed and ran the test program bears the larger share of responsibility for it. Our own program should be judged by the developer's failures in that case, not by the operator's.

The Record Used

This memo relies on the federal highway accident report on the crash (National Transportation Safety Board, 2019), in which a Volvo XC90 operated by Uber's Advanced Technologies Group under its developmental automated driving system struck and killed a pedestrian walking a bicycle across a road at night. The NTSB report is the most complete public record of the decisions that preceded the crash. Names of the individuals involved are omitted here.

The Chain, Step by Step

1. Design: when the automated system was in control, the vehicle's factory-installed collision avoidance features, including automatic emergency braking, were disabled to prevent conflicts with the developmental system. The developmental system itself was designed so that emergency braking maneuvers were not enabled while the computer drove; the operator was expected to intervene.

2. Perception and action suppression: the system first detected the pedestrian about 5.6 seconds before impact but repeatedly changed how it classified her and did not predict her path as crossing the vehicle's lane. When it determined that a crash was imminent, about 1.2 seconds before impact, a built-in action suppression of one second delayed any response, leaving the operator to act.

3. Staffing: the developer had moved in late 2017 from two operators per vehicle to one, removing the second person who had monitored the system and logged events.

4. Oversight: the developer did not adequately monitor whether its operators were watching the road, despite knowing that people supervising automation tend to lose attention over long periods.

5. The final seconds: the single operator was looking down, according to the NTSB, at a personal cell phone that was streaming video, and looked up too late to prevent the crash.

The NTSB found that the probable cause was the operator's failure to monitor the road because of distraction by her phone. It found that the developer's inadequate safety risk assessment procedures, ineffective oversight of operators and lack of mechanisms to address automation complacency contributed to the crash, along with the pedestrian's crossing outside a crosswalk while impaired and inadequate state oversight of testing.

The Moral Crumple Zone

Elish (2019) introduced the term moral crumple zone for the way responsibility collapses onto the human nearest an automated system's failure, much as a car's crumple zone absorbs the force of a crash, while the system and its designers are protected. The Tempe case fits her description closely. The operator was the only person criminally charged; prosecutors declined to charge the developer. Public attention centered on the phone. Yet steps 1 through 4 were decisions made by the company, and each one made the operator's lapse more likely to be fatal: a vehicle that could not brake on its own, a system built to hand emergencies to a person, a second person removed and no monitoring of the one who remained.

The Problem of Many Hands

Thompson (1980) described the problem of many hands: when many people contribute to an outcome in a large organization, it becomes difficult to identify any one of them as responsible, and officials can each disclaim responsibility. The rival difficulty here is real. No single engineer decided to build a system that would kill a pedestrian; braking was disabled by one team, staffing set by another and operator oversight left to a third. If responsibility is spread across all of them, it may seem to rest on none.

Resolving the Two

The two concepts point to the same answer once the organization is treated as the agent. Thompson's own remedy was to hold officials responsible for the structures and policies they create, not only for direct acts. The developer, as an organization, chose a design that relied on one human to catch every failure, then removed the conditions that would help that human do so. The operator was the last person who could have stopped the outcome. The developer was the party that made the outcome depend on her.

What This Means for Our Program

Our test program uses two operators per vehicle and keeps factory braking active. Both choices should be set down in policy, beyond the reach of a later cost review. We should also add camera-based monitoring of operator attention, reviewed weekly, and require any change in staffing or in the handoff of emergency responses to pass the safety committee. If our own program ever relies on a single person to catch everything the system misses, the Tempe record tells us where responsibility will truly lie, whatever a charging decision says.

References

Elish, M. C. (2019). Moral crumple zones: Cautionary tales in human-robot interaction. Engaging Science, Technology, and Society, 5, 40-60. https://doi.org/10.17351/ests2019.260

National Transportation Safety Board. (2019). Collision between vehicle controlled by developmental automated driving system and pedestrian, Tempe, Arizona, March 18, 2018 (Highway Accident Report NTSB/HAR-19/03). https://www.ntsb.gov/investigations/Pages/HWY18MH010.aspx

Thompson, D. F. (1980). Moral responsibility of public officials: The problem of many hands. American Political Science Review, 74(4), 905-916. https://doi.org/10.2307/1954312