MGMT 8705M Week 4 Indicator Critique Example

What Billable Hours Leave Uncounted and What Leaks In: An Indicator Critique Mapping a Consulting Firm's Most Trusted Number Against the Performance It Claims to Record, Naming the Deficient and Contaminated Regions and Asking Whether a Second Indicator Would Help

Student Name

Walden University

MGMT 8705M: Organizational Behavior Performance and Evaluation

Instructor Name

Month Day, Year

What Billable Hours Leave Uncounted and What Leaks In: An Indicator Critique Mapping a Consulting Firm's Most Trusted Number Against the Performance It Claims to Record, Naming the Deficient and Contaminated Regions and Asking Whether a Second Indicator Would Help

The Construct the Number Claims

At Hollins & Reyes, a composite management consulting firm with about 300 consultants, annual reviews give heavy weight to billable hours, the hours a consultant records against client projects. The firm's performance guide defines consultant performance as "delivering high-quality client work, developing junior colleagues and building the firm's practice." Billable hours are treated as the most objective measure of the first part and, informally, of the whole. This critique maps the construct against what the indicator records.

The Overlap

Picture two circles. One is the construct, consultant performance as the guide defines it. The other is what billable hours record. Where they overlap, hours capture real performance: time spent delivering work clients paid for. The region of the construct outside the overlap is deficiency, performance the number never sees. The region of the indicator outside the overlap is contamination, influences that enter the number without being performance. Austin and Villanova (1992), reviewing the long history of the criterion problem in performance measurement, described deficiency and contamination as the two persistent ways measures depart from the performance they are meant to capture.

What Performance Holds That Hours Miss

Task performance, the core technical work of a job, can be told apart from contextual performance, the behavior that supports colleagues and the organization, and supervisors weigh both when they judge overall performance (Motowidlo & Van Scotter, 1994). Billable hours capture only part of task performance and almost none of contextual performance. Training junior consultants bills to no client. Writing proposals for new work, which builds the practice, is charged to an internal code. Improving the firm's methods, mentoring and reviewing colleagues' work all fall outside the count. Two of the three parts of the firm's own definition are therefore largely invisible to its main indicator.

What Leaks Into the Number

Several influences enter billable hours without reflecting a consultant's performance. The partner who staffs projects decides how many billable hours a consultant can accumulate, so the number partly measures staffing decisions. Client budgets cap billable time on some projects regardless of the work needed. And slow work bills more than fast work: a consultant who completes an analysis in twenty hours records fewer billable hours than one who takes forty, though the faster consultant has performed better.

The Assigner as an Unseen Rater

The staffing partner deserves particular attention. Because assignments drive hours, the partner's judgments about who is capable, or whom they prefer to work with, enter the indicator invisibly. Billable hours appear objective but carry a subjective judgment inside them, made by someone who never fills in a rating form.

What the Research Says About Objective and Subjective Measures

Bommer et al. (1995), in a meta-analysis of studies with both objective and subjective measures of employee performance, found a corrected mean correlation of about .39 and concluded that the two kinds of measure should not be used interchangeably. Their finding warns against treating billable hours, the firm's objective measure, as a substitute for partners' judgments of quality and development, or the reverse.

Would a Second Indicator Help?

A second indicator could reduce the deficiency: for example, a count of junior consultants a person supervised whose reviews improved, or hours charged to proposal work that won engagements. But each new count carries its own contamination and invites its own gaming. The more useful change may be to stop treating hours as performance and report them as what they are: a record of staffing and client billing, one input among several in a review that also asks peers and juniors what the consultant contributed.

What the Firm Could Do Without a New Indicator

Some contamination could be reduced without adding a measure. Partners could rotate staffing so that no consultant's hours depend on one partner's preferences, and reviews could compare hours only among consultants on similar projects. Deficiency is harder to fix by adjusting hours, since the missing work never appears in them. That is the strongest argument for adding qualitative evidence from peers and juniors to every review.

Limits

The critique relies on the firm's guide and on general research; it has not measured how much each source of contamination affects hours at Hollins & Reyes. Comparing hours with independent quality ratings across a year of projects would test it.

References

Austin, J. T., & Villanova, P. (1992). The criterion problem: 1917-1992. Journal of Applied Psychology, 77(6), 836-874. https://doi.org/10.1037/0021-9010.77.6.836

Bommer, W. H., Johnson, J. L., Rich, G. A., Podsakoff, P. M., & MacKenzie, S. B. (1995). On the interchangeability of objective and subjective measures of employee performance: A meta-analysis. Personnel Psychology, 48(3), 587-605. https://doi.org/10.1111/j.1744-6570.1995.tb01772.x

Motowidlo, S. J., & Van Scotter, J. R. (1994). Evidence that task performance should be distinguished from contextual performance. Journal of Applied Psychology, 79(4), 475-480. https://doi.org/10.1037/0021-9010.79.4.475