DDHA 8303 Week 3 Article Appraisal Example

Thirteen Percent More Output From Volunteers in a Shanghai Call Center: An Appraisal of Bloom and Colleagues' Work-From-Home Experiment and How Far Its Conclusion Travels

Student Name

Walden University

DDHA 8303: Research Methods for Healthcare Administrators

Instructor Name

Month Day, Year

Thirteen Percent More Output From Volunteers in a Shanghai Call Center: An Appraisal of Bloom and Colleagues' Work-From-Home Experiment and How Far Its Conclusion Travels

Why This Article

A director weighing whether to bring remote coders back to the office will likely be shown one study more than any other: the randomized work-from-home experiment that Bloom et al. (2015) ran at Ctrip, a large Chinese travel agency. It is widely cited, since randomized trials of remote work inside real firms are rare. This appraisal asks what the study's procedure permits its authors to claim and where the claim, as it is usually repeated, goes further than the design allows.

What the Study Did

The study took place in the airfare and hotel booking departments of Ctrip's call center in Shanghai. Employees in those departments were asked whether they wanted to work from home four days a week. Among those who volunteered and met eligibility conditions, which included tenure, a private room at home and broadband access, workers with even-numbered birthdays were assigned to work from home and those with odd-numbered birthdays stayed in the office. The experiment ran for nine months. Performance was measured mainly from the firm's own records of calls handled and minutes worked, data that existed regardless of the study. The authors also examined attrition, promotion and self-reported satisfaction.

The authors reported that home workers' performance rose by about 13%. They attributed most of the gain to working more minutes per shift, through fewer breaks and sick days, and the remainder to handling more calls per minute, which they linked to a quieter environment. Attrition among home workers fell by about half. After the experiment, Ctrip offered the option to the whole firm, and the authors reported that some home workers chose to return to the office, so that later performance gains reflected workers selecting the arrangement that suited them.

What the Design Permits

Random assignment by birthday is the study's main strength. Because the volunteers were divided by a rule unrelated to their ability or motivation, differences that appeared between the groups during the experiment can reasonably be attributed to working from home rather than to who chose it. Performance data drawn from existing company systems also reduce the risk that participants' knowledge of the study shaped how outcomes were recorded. Within its population, the causal claim that working from home increased output is well supported.

Where the Claim Outruns the Method

Three threats limit how far that claim travels.

The first is the sample. Only volunteers were randomized. The study therefore estimates the effect of working from home on people who wanted to, had suitable homes and met tenure requirements. It cannot say what would happen to employees required to work from home, or to those who lacked a quiet room. The authors' own observation that some home workers later returned to the office supports this point: the arrangement suited some volunteers and not others. When the result is summarized as "working from home raises productivity by 13%," the conditional part of the finding, among volunteers with suitable homes, is lost.

The second is the outcome. Call center performance at Ctrip was largely countable: calls taken, minutes on the phone, orders placed. The study shows that home workers did more of this measurable work. It is less informative about quality, because quality in a booking call is harder to observe than volume. For an administrator interested in coding accuracy, this is the central limitation. Coding output and coding accuracy can move in opposite directions, since a coder who works faster may make more errors. A study that measures volume well does not tell us what happens to accuracy.

The third is the setting. The work was performed by individuals, required little collaboration and was supported by a mature call center system. The study took place in one firm, in one city, at one point in time. Other researchers who have reviewed the telecommuting literature note that effects vary with job type and with how much of the week is spent away from the office (Allen et al., 2015). Bloom et al. studied an arrangement of four days at home; results for fully remote work, which is what many health systems adopted for coders, may differ, particularly in the coworker relationships that Gajendran and Harrison (2007) found high-intensity telecommuting can harm.

A fourth concern, contamination between groups, is weaker than it might appear. Office workers knew colleagues had been sent home, which might have changed their own effort. This threat seems smaller than the others, because the office group's output came from the same routine records, and a decline large enough to explain a 13% gap would have been visible in them.

Is the Authors' Own Account of Limits Adequate?

The authors acknowledge that the results come from volunteers and from a particular kind of work, and they discuss the role of selection after the experiment ended. That discussion is candid. What it does not do is warn readers sharply enough that the outcome measure is almost entirely about quantity. Given how often the study is cited in debates about professional work, a plainer statement that the experiment cannot speak to work whose quality is hard to observe would have helped readers apply it responsibly.

What an Administrator Can Take From It

An administrator can reasonably take three things from this study. First, for individual, measurable work, allowing volunteers to work from home does not necessarily reduce output and may increase it. Second, much of the gain may come from more working time, not better work. Third, letting workers choose may matter, since the arrangement does not suit everyone.

What would be unwarranted is concluding that remote coders will be more accurate, or even as accurate, because Ctrip's call center workers produced more. The study did not measure accuracy in a sense that applies to coding, did not include employees who were required to work from home and did not test fully remote work. For Halvorsen's question, the article justifies taking remote work seriously as a workable option. It does not answer whether coding accuracy changed, which requires the system's own audit data.

Conclusion

Bloom et al. (2015) report a well-designed experiment whose internal validity is strong. Its conclusion is sound for volunteers doing countable call center work at one firm. Repeated as a general claim that remote work raises productivity, it outruns the design. For a health system concerned about coding accuracy, it is useful background and no substitute for a local study.

References

Allen, T. D., Golden, T. D., & Shockley, K. M. (2015). How effective is telecommuting? Assessing the status of our scientific findings. Psychological Science in the Public Interest, 16(2), 40-68. https://doi.org/10.1177/1529100615593273

Bloom, N., Liang, J., Roberts, J., & Ying, Z. J. (2015). Does working from home work? Evidence from a Chinese experiment. The Quarterly Journal of Economics, 130(1), 165-218. https://doi.org/10.1093/qje/qju032

Gajendran, R. S., & Harrison, D. A. (2007). The good, the bad, and the unknown about telecommuting: Meta-analysis of psychological mediators and individual consequences. Journal of Applied Psychology, 92(6), 1524-1541. https://doi.org/10.1037/0021-9010.92.6.1524