MGMT 8825M Week 10 Deployment Analysis Example

Enabled on a Tuesday, Noticed Three Weeks Later: A Deployment Analysis of an Outdoor Retailer's Generative Support Assistant That Promised Refunds Outside Policy, Naming Each Point Where Review Could Have Happened and Why It Did Not

Student Name

Walden University

MGMT 8825M: Ethics of Artificial Intelligence and Autonomous Systems

Instructor Name

Month Day, Year

Enabled on a Tuesday, Noticed Three Weeks Later: A Deployment Analysis of an Outdoor Retailer's Generative Support Assistant That Promised Refunds Outside Policy, Naming Each Point Where Review Could Have Happened and Why It Did Not

The Deployment

Ridgeback Outfitters, a composite online retailer of camping and climbing gear with about 140 employees in Boulder, Colorado, handles customer questions through a help desk platform bought from a large software vendor. In February 2026 the vendor released a generative assistant feature that drafts and sends answers to customer chats, drawing on the retailer's help articles and past tickets. Ridgeback's customer support team lead enabled the feature for all chat traffic the following week. Three weeks later, the finance team noticed a rise in refund requests on used and long-held items, several of which quoted chat transcripts in which the assistant had told customers they could return any item for a full refund within a year. Ridgeback's policy allows refunds within 30 days for unused items.

The Timeline and the Missed Review Points

Four chances for review were missed before the fifth caught the problem. None required unusual foresight. Each was missed for an ordinary reason: no owner, no test, an assumption about the vendor or no rule.

Table 1

Deployment Timeline and Review Points

Dates are reconstructed from the admin console log, chat records and the finance team's ticket.

An Audit Across the Life of the System

Raji et al. (2020) proposed an internal audit framework for AI systems that runs across the life of a system rather than at a single gate. Its stages are scoping, which sets out the system's intended use and the risks it raises; mapping, which identifies the people and processes involved; artifact collection, which gathers documentation; testing, which checks the system against its risks; and reflection, which weighs results and decides what to change. They argued that responsibility gaps in AI systems arise when no one inside the organization owns these stages.

Applied to Ridgeback, the framework shows where each stage would have caught the failure. Scoping would have identified customer promises about money as a high-risk use. Mapping would have shown that the assistant's answers go straight to customers with no person in between. Artifact collection would have surfaced the vendor's note that the assistant could draw on past tickets, including outdated ones. Testing would have meant asking the assistant twenty questions about refunds before it went live. Reflection would have set a rule for when to turn it off.

A Structure the Retailer Could Have Used

The federal AI Risk Management Framework (National Institute of Standards and Technology, 2023) was written to help organizations of any size manage the risks of AI systems. It is organized around four functions: govern, which sets policies, roles and accountability; map, which establishes the context and identifies risks; measure, which assesses and tracks them; and manage, which prioritizes and acts on them. For a retailer of Ridgeback's size, the framework's main value would have been in the govern function. A one-page policy assigning an owner to any AI feature that speaks to customers, and requiring a test before it goes live, would have closed the first two gaps. Measure and manage could have been as simple as a weekly review of a sample of the assistant's answers and an automatic flag on any answer mentioning refunds.

Where Accountability Lands

Ridgeback's first response, in an email to customers, was that the assistant had "made an error" and that its statements did not reflect company policy. A published case shows why that position is weak. In a 2024 decision (Moffatt v. Air Canada, 2024), a British Columbia tribunal considered a customer who had relied on the airline's website chatbot, which told him he could apply for a bereavement fare after travel. The airline argued that the chatbot was a separate legal entity responsible for its own actions. The tribunal rejected the argument, finding that the airline was responsible for all the information on its website, whether it came from a static page or a chatbot, and it awarded the customer damages.

The decision is not binding on a Colorado retailer, but its reasoning is the one most customers, and most courts, would apply. The assistant spoke for Ridgeback. Ridgeback honored the 41 refund promises the assistant had made, at a cost of about $9,300, which was the right decision both ethically and commercially.

What Should Change

Three changes would prevent a repeat. Any AI feature that communicates with customers should have a named owner and should not be enabled without a recorded test. Answers touching money, safety or legal terms should go to a person before they are sent. And vendor release notes should be read by someone with authority to delay a feature. The vendor carries part of the responsibility too. It released a feature that could send answers to customers by default, drew on old tickets without warning that they might contradict current policy, and announced the change in a console notice few administrators read. Ridgeback should ask the vendor to ship such features switched off, with a test mode that shows draft answers to staff before any reach a customer. None of these steps is expensive. Together they would have cost less than the refunds.

References

Moffatt v. Air Canada, 2024 BCCRT 149 (2024). https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/2024bccrt149.html

National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0) (NIST AI 100-1). https://doi.org/10.6028/NIST.AI.100-1

Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 33-44). Association for Computing Machinery. https://doi.org/10.1145/3351095.3372873