DBAX 9104 Week 10 Defense Rehearsal Example

Eight Objections a Committee Will Raise: A Written Defense Rehearsal for a Correlational Study of Security-Awareness Training and Phishing-Simulation Failures at Small Accounting Firms, Hardest Question First

Student Name

Walden University

DBAX 9104: Business Research Project Stage 4

Instructor Name

Month Day, Year

Eight Objections a Committee Will Raise: A Written Defense Rehearsal for a Correlational Study of Security-Awareness Training and Phishing-Simulation Failures at Small Accounting Firms, Hardest Question First

How the Rehearsal Was Built

Most defense questions can be predicted from the document itself. I found these eight by reading my study as a skeptical committee member would, looking for distances between what chapter one worried about and what chapter three measured, between what the sample was and what the conclusions describe, and between the data and the verbs used about them. The study examined 61 accounting firms with 5 to 40 employees in the Southeast and asked whether the annual hours of security-awareness training per employee were related to the rate at which employees clicked links in simulated phishing emails sent by the firms' managed IT vendor. Each objection below has four parts: the question as a committee member might ask it, where the document already addresses it, a spoken answer of two or three sentences, and what the answer concedes.

Objection 1: Does the Study Address the Problem at All?

Question: "Chapter one says the problem is financial loss from breaches. You measured clicks on fake emails. How do your findings speak to the problem?"

Where addressed: Closing chapter, limitations, p. 112; method, construct section, p. 58.

Answer: "Clicks on simulated phishing are a measure of susceptibility, not of loss. The study shows training hours were associated with fewer clicks, and the closing chapter limits its claims to susceptibility. Whether fewer clicks mean fewer losses is a further link the study did not test."

Concedes: The distance between the problem and the measure is real. Shadish et al. (2002) describe this as a question of construct validity, whether the operations of a study represent the constructs it claims to address. Chapter one now states that the study addresses one step in the path from a phishing email to a financial loss.

Objection 2: Volunteer Firms

Question: "Firms chose to take part. Aren't the firms that train more also more careful in other ways?"

Where addressed: Limitations, p. 113.

Answer: "Yes, that is possible, and the design cannot rule it out. Firms that invest in training may also have stricter email filtering or a culture of caution."

Concedes: Self-selection limits the conclusion to association. The closing chapter never says training reduced clicks.

Objection 3: The Vendor Built the Measure

Question: "Your click rates come from the IT vendor's simulation reports. How do you know the simulations were comparable across firms?"

Where addressed: Method, data sources, p. 61.

Answer: "All 61 firms used the same vendor and the same quarterly simulation templates, which the vendor confirmed in writing. Difficulty was therefore similar across firms, though not identical across quarters."

Concedes: Difficulty varied by quarter, which is why the analysis used annual averages.

Objection 4: Direction of the Association

Question: "Could firms with high click rates have increased training in response?"

Where addressed: Results, p. 84; limitations, p. 113.

Answer: "That is possible. Training hours and click rates were measured over the same year, so the order is unknown. A firm that had a bad quarter might well have added training."

Concedes: Reverse ordering would bias the association toward zero or even positive, so the negative association found may understate any training effect, but the study cannot say.

Objection 5: Does Training Type Matter?

Question: "You treat an hour of video the same as an hour of live coaching. Is that reasonable?"

Where addressed: Delimitations, p. 14.

Answer: "The study measured hours, not format, because firms' records did not distinguish them reliably. Kumaraguru et al. (2010) found that embedded, interactive training improved resistance to phishing, which suggests format matters."

Concedes: Format is a plausible moderator the study could not test.

Objection 6: Who Clicks

Question: "Were some employees more likely to click regardless of training?"

Where addressed: Discussion, p. 101.

Answer: "Sheng et al. (2010) found differences in phishing susceptibility by age and gender in an online study. My data were at the firm level, so I could not examine individual differences."

Concedes: Firm-level data hide which employees drive the click rate.

Objection 7: Sample Size

Question: "Sixty-one firms is a small sample for a regression with four controls. Is the model stable?"

Where addressed: Method, power analysis, p. 55; results, diagnostics, p. 86.

Answer: "The power analysis required 55 firms for a medium effect. Diagnostics showed no influential cases, and the training coefficient kept its sign and size when each firm was dropped in turn."

Concedes: The study could not detect small effects of the controls.

Objection 8: What Should a Firm Do?

Question: "If I ran a ten-person firm, what would you tell me?"

Where addressed: Recommendations, p. 115.

Answer: "Firms with more training hours had lower click rates in this sample. A firm could track its own click rate before and after adding training. The study does not show how many hours are enough."

Concedes: No dose recommendation is supported.

Revisions the Rehearsal Forced

Writing the answers exposed three passages that did not say what I needed them to say. The construct passage on page 58 did not mention the gap between susceptibility and loss; it now does. The limitations section did not discuss reverse ordering; a paragraph was added. And the recommendations section had suggested "at least four hours" of training a year, a figure the data cannot support; it was removed.

References

Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L. F., & Hong, J. (2010). Teaching Johnny not to fall for phish. ACM Transactions on Internet Technology, 10(2), 1-31. https://doi.org/10.1145/1754393.1754396

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

Sheng, S., Holbrook, M., Kumaraguru, P., Cranor, L. F., & Downs, J. (2010). Who falls for phish? A demographic analysis of phishing susceptibility and effectiveness of interventions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 373-382). Association for Computing Machinery. https://doi.org/10.1145/1753326.1753383