DDBA 8307B Week 5 Group Comparison Paper Example

Clear Enough to Be Significant, Too Small to Pay for a Refit: Comparing Average Basket Value Between Hardware Stores With a New Checkout Layout and Stores Left Unchanged, and Asking What the Result Would Change for the Decision

Student Name

Walden University

DDBA 8307B: Quantitative Business Data Analysis Using SPSS

Instructor Name

Month Day, Year

Clear Enough to Be Significant, Too Small to Pay for a Refit: Comparing Average Basket Value Between Hardware Stores With a New Checkout Layout and Stores Left Unchanged, and Asking What the Result Would Change for the Decision

The Decision and the Data

A composite chain of 26 hardware stores in the Pacific Northwest refitted the checkout areas of 12 stores last spring, adding impulse displays of small tools, batteries and fasteners along a single queue that feeds all registers. The other 14 stores kept separate lines at each register. The regional manager must decide whether to refit the remaining 14 stores at about $85,000 each. The data are all transactions over eight weeks in late summer: 48,200 in the refitted stores and 51,900 in the unchanged stores. The question is whether average basket value differs between the two groups and, if it does, whether the difference is large enough to justify the cost.

The Comparison

Baskets in the refitted stores averaged $0.68 more. Welch's t-test treating each transaction as an observation gives t = 3.43, p < .001, and the plausible range for the gap, at 95 percent confidence, runs from about $0.29 to $1.07. By the usual standard, the difference is statistically significant.

Table 1

Basket Value by Store Group, Eight Weeks

Computed in SPSS from the point-of-sale export; returns excluded.

How Large Is the Difference?

The standardized effect size, Cohen's d, is about 0.02, a tiny fraction of the spread in basket values. Sullivan and Feinn (2012) emphasize that a p value says only whether a difference is likely to be zero, not whether it matters; with nearly 100,000 transactions, almost any difference will be significant. The useful question is what $0.68 a basket is worth. A typical store in the chain handles about 2,000 transactions a week. At $0.68 more per transaction, that is about $1,360 more revenue a week, or about $70,700 a year. At the chain's gross margin of about 35 percent, the gain is about $24,700 a year in gross profit per store, so an $85,000 refit would take about three and a half years to pay back, before any added labor or display costs. The impulse displays also need restocking, which the store managers estimate at two to three staff hours a week, a cost the estimate leaves out.

Why the Estimate Is Less Certain Than It Looks

The t-test treated each transaction as independent. They are not: transactions in the same store share that store's location, customers and staff, so the real unit of comparison is closer to the store than the transaction. With only 12 and 14 stores, a store-level comparison of average baskets has much less power. Comparing store means, refitted stores averaged $38.70 and unchanged stores $37.90, a difference of $0.80, but with t(24) = 0.62 and p = .54, the store-level data cannot distinguish the difference from zero. Cumming (2014) recommends reporting intervals and thinking about precision; the transaction-level interval is too narrow because it ignores how transactions cluster within stores.

There is a second limit. The 12 stores were not chosen at random; the regional manager refitted stores whose leases were up for renewal, which tend to be older stores in established neighborhoods. Any difference may reflect the stores, not the layout.

Why Not Simply Report the Significant Result

It would have been easy to report that the refit increased basket value significantly and stop there. Many internal reports do. But significance with nearly 100,000 transactions says little about whether the effect is worth money, and the transaction-level test overstates precision by ignoring which store each transaction came from. Reporting the effect in dollars, its payback and the store-level result gives the manager the three pieces of information the decision depends on: how large the gain might be, how long it would take to pay for itself and how sure the data allow anyone to be.

What the Result Would Change

For the decision, the result changes little. Even taking the transaction-level estimate at face value, the payback is about three and a half years, longer than the chain's usual two-year standard for store improvements. And the store-level analysis cannot rule out no difference at all. Field (2018) notes that statistical tests answer narrow questions; the manager's question, whether the refit pays, needs the dollar estimate and its uncertainty. A better next step than a chainwide refit would be to refit four more stores chosen at random and compare them with four matched stores over a full quarter.

References

Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7-29. https://doi.org/10.1177/0956797613504966

Field, A. (2018). Discovering statistics using IBM SPSS Statistics (5th ed.). SAGE Publications.

Sullivan, G. M., & Feinn, R. (2012). Using effect size, or why the P value is not enough. Journal of Graduate Medical Education, 4(3), 279-282. https://doi.org/10.4300/JGME-D-12-00156.1