DBAX 9102 Week 7 Research Design Options Example

Four Designs Against One Remainder: Lining Up an Experiment, a Branch Comparison, a Multilevel Records Study and a Case Study for Testing Whether Mandated Coaching Frequency Relates to Quota Attainment

Student Name

Walden University

DBAX 9102: Business Research Project Stage 2

Instructor Name

Month Day, Year

Four Designs Against One Remainder: Lining Up an Experiment, a Branch Comparison, a Multilevel Records Study and a Case Study for Testing Whether Mandated Coaching Frequency Relates to Quota Attainment

What the Review Has Already Decided

Once the review has spoken, most designs are already out of the running. The open remainder from last week requires four things: branch sales managers in industrial distribution, coaching frequency logged in company systems, coaching skill rated by salespeople, and several quarters of quota results drawn from company systems, all under a policy that sets a frequency target. Any design that cannot deliver those four is ruled out before its other merits are considered. This paper sets four candidate designs against those requirements, at the regional distributor where I work, with fourteen branches, eleven sales managers and sixty-four outside salespeople.

The Four Candidates

Table 1

Candidate Designs Against the Remainder's Requirements

Feasibility judged from conversations with the vice president of sales and the sales operations team.

Weighing the Candidates

The field experiment would give the clearest answer. Antonakis et al. (2010) argue that randomized designs are the most reliable way to support causal claims in leadership research. But leadership will not exempt any manager from the coaching policy, so random assignment is not available. A design that cannot be run cannot be chosen.

The branch comparison has appeal because the policy began at a known date. However, coaching sessions were not logged before the policy, so frequency before the change cannot be measured, and the same year saw a price increase and two new product lines. Shadish et al. (2002) describe how such concurrent events, which they call history threats, undermine simple before-and-after comparisons.

The multilevel records study meets all four requirements. It can relate each salesperson's logged coaching frequency and rated coaching skill to recorded quota attainment, accounting for the fact that salespeople share managers. Dahling et al. (2016) used a multilevel design of this general kind in a sales organization. Its weakness is causal reach: it can show association and control for measured differences between territories, but not rule out unmeasured ones.

The case study would explain how managers turn a frequency target into actual coaching, which matters for interpreting any result. But it cannot estimate the relationship the remainder asks about.

The Likely Choice

The multilevel records study is the only design that meets every requirement and can be run. The case study's strength, explaining mechanisms, could be added in a limited form, through a few follow-up interviews with managers, without changing the main design. Next week's commitment paper will settle the choice and state what it gives up.

Designs Considered and Not Listed

Two other designs were considered briefly. A survey-only study, asking salespeople to rate both their manager's coaching and their own performance, would be cheap and fast, but it would repeat the ratings-based design that the disagreement analysis identified as the likely source of inflated results. A study across several distributors would improve generalizability, but I have no access to other firms' coaching logs or quota records, and their coaching policies would differ. Neither meets the remainder's requirements, so neither appears in the table. Recording them here shows that the four candidates were chosen from a wider field rather than assembled to make one design look inevitable, which a committee is entitled to ask about.

What the Comparison Shows About the Remainder

Laying the designs side by side also tests the remainder itself. If no design could have met all its requirements, the remainder would be too ambitious. One can, which suggests the remainder is both specific and studyable. It also confirms that the remainder was not written to fit a design chosen in advance, since the comparison could have ruled every design out.

References

Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. Leadership Quarterly, 21(6), 1086-1120. https://doi.org/10.1016/j.leaqua.2010.10.010

Dahling, J. J., Taylor, S. R., Chau, S. L., & Dwight, S. A. (2016). Does coaching matter? A multilevel model linking managerial coaching skill and frequency to sales goal attainment. Personnel Psychology, 69(4), 863-894. https://doi.org/10.1111/peps.12123

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.