DDHA 8303 Week 10 Analysis Plan Example

Committed Before the First Record Is Opened: An Analysis Plan for Audit Agreement, Engagement Scores and Interviews in a Study of Remote Inpatient Coders

Student Name

Walden University

DDHA 8303: Research Methods for Healthcare Administrators

Instructor Name

Month Day, Year

Committed Before the First Record Is Opened: An Analysis Plan for Audit Agreement, Engagement Scores and Interviews in a Study of Remote Inpatient Coders

Purpose of the Plan

This plan fixes how each of the study's three questions will be analyzed before any data are extracted. The audit records already exist, and the research team could look at them at any time, which makes advance commitment more important rather than less. A plan written after seeing the data could be shaped, even unintentionally, to fit whatever the data showed. The plan will be filed with the research office and dated before the extract is requested.

Questions and Procedures

Research question 1 asks whether the share of audited inpatient accounts on which a coder's DRG matched the auditor's changed for coders after their 2024 move to home-based work, compared with coders in the same months who had not moved or never did. It is answered with a staggered difference-in-differences analysis of monthly audit records.

Research question 2 asks whether work engagement scores differ between coders working remotely and those working onsite or hybrid, and whether engagement is associated with the change in a coder's agreement rate. It is answered with regression on survey data linked to audit results.

Research question 3 asks how inpatient coders describe resolving uncertain coding decisions, and how remote and onsite coders' accounts differ. It is answered with framework analysis of interview transcripts.

Variables

Table 1

Variables and Their Measurement Levels

Hospital of origin is held in the research file for screening only and is never used in a reported analysis, to protect coders at small sites.

Descriptive Summaries

Before any inferential procedure, the analysis will describe the coders by move wave: number, tenure, credential mix and audit agreement during 2023. It will plot the monthly agreement rate for each wave across all 36 months, with the two move dates marked. These plots will be produced before any model is fitted and included in the report regardless of what they show.

Inferential Procedure for Question 1

The outcome is the monthly agreement proportion for each coder, weighted by the number of accounts audited that month. The main estimate will use a group-time approach (Callaway & Sant'Anna, 2021), which estimates the effect for each move wave in each month after the move, comparing it only with coders still in the office at that point, whether they moved later or not at all. These estimates will then be combined into an overall average effect and into an event-time profile showing the effect at each month relative to the move. This choice responds to the bias that can arise when ordinary two-way fixed effects regression is applied to staggered timing.

Three assumptions are required, each with a check. First, parallel trends: agreement in the groups would have changed similarly without the move. The check is the event-time profile for the months before each move; estimates near zero, individually and jointly, support the assumption, and estimates that drift would be reported as a threat. Wing et al. (2018) recommend this kind of pre-period examination as a standard practice. Second, no anticipation: coders did not change their work before moving. Because coders were told of the moves about six weeks in advance, the month before each move will be excluded in a sensitivity analysis. Third, no spillover: one coder's move did not change another's accuracy. This cannot be fully tested, because remote coders may have consulted onsite colleagues less, which could affect both groups, and the report will discuss it as a limitation.

Standard errors will be clustered by coder. A secondary analysis will fit a logistic mixed model to account-level agreement, with a random intercept for each coder and fixed effects for month, as a check on whether the main estimate depends on the modeling approach.

Inferential Procedure for Question 2

Engagement totals will be compared between remote and onsite or hybrid coders using linear regression, adjusting for tenure. The residuals will be checked for normality with a quantile plot and for equal variance by group. As committed in the instrument appraisal, internal consistency will be computed separately for each group, and if it differs markedly, the comparison will be reported without interpretation. The association between engagement and each coder's change in agreement will be examined with a correlation and labeled as exploratory, since the study was not designed to test it and has limited power.

Coding Approach for Question 3

Interview transcripts will be analyzed with the framework method described by Gale et al. (2013), which suits studies that combine predefined questions with openness to unexpected themes. The analysis moves through familiarization with the transcripts, initial coding, development of a working analytical framework, indexing of all transcripts against the framework, charting the indexed data into a matrix of coders by themes and interpretation.

Codes will come from two sources. Some will be drawn in advance from published accounts of where coding errors arise, including documentation gaps, the query process with physicians and coder judgment. Others will be developed from the data as unexpected practices emerge. Two analysts will independently code the first four transcripts, compare their coding line by line and revise the codebook until they agree on how each code applies. Agreement will be established through discussion and recorded in a codebook log, not reported as a single statistic, because the purpose is a shared understanding of the codes rather than a score. After that, one analyst will code the remaining transcripts and a second will review every fifth one. The matrix will be charted with remote and onsite coders in separate columns so that differences are visible.

Missing Data and Attrition

Months in which a coder had fewer than ten audited accounts will be flagged, and the main analysis will be repeated without them. Months with no audits will be treated as missing, not as zero agreement. Coders who left during the period remain in the analysis up to their departure; a sensitivity analysis will use only coders present for all 36 months, so that any difference between the two results can be reported as a possible effect of attrition. For the survey, respondents will be compared with the full frame on tenure, wave and 2023 agreement, and no imputation will be used. If fewer than twelve interviews are completed, the report will say that the qualitative findings may not capture the full range of practices.

Integration

After each branch is analyzed separately, results will be brought together in a table setting each wave's estimated change in agreement beside the themes from that wave's interviews. Disagreement between the branches will be reported as a finding, not resolved by preferring one.

References

Callaway, B., & Sant'Anna, P. H. C. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200-230. https://doi.org/10.1016/j.jeconom.2020.12.001

Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13, Article 117. https://doi.org/10.1186/1471-2288-13-117

Wing, C., Simon, K., & Bello-Gomez, R. A. (2018). Designing difference in difference studies: Best practices for public health policy research. Annual Review of Public Health, 39, 453-469. https://doi.org/10.1146/annurev-publhealth-040617-013507