PHLT 8270 Week 5 Data Quality Appraisal Example

Measuring Data Quality as Distance From Named Standards: Field Completeness, Report Timeliness and Code Validity in a Composite State's Carbapenem-Resistant Organism Reports

Student Name

Walden University

PHLT 8270: Health Informatics and Surveillance

Instructor Name

Month Day, Year

Measuring Data Quality as Distance From Named Standards: Field Completeness, Report Timeliness and Code Validity in a Composite State's Carbapenem-Resistant Organism Reports

The Approach

Data quality is often described in general terms: the reports are mostly complete, or timeliness is improving. Such statements cannot guide repair, because they do not say complete enough for what or timely compared with what. This appraisal measures the composite state's carbapenem-resistant organism reports against three named standards: the reporting deadline in the state's communicable disease rule, the window the healthcare-associated infections team needs to start containment and the fields the state's electronic laboratory reporting guide marks as required. Each measure is reported as the distance between practice and the standard. Dixon et al. (2013) describe completeness, timeliness and accuracy as the dimensions of data quality that most affect surveillance practice, and this appraisal applies them to one system. All figures are composite.

Standard 1: Required Fields

The state's reporting guide marks eight fields as required for carbapenem-resistant isolates. Completeness was measured for each field separately across the 412 reports received in one year.

Reporting a single completeness figure across all fields would have shown about 82%, hiding the fields that matter most for containment. The two weakest, facility type and carbapenemase testing, are the two the containment response depends on.

Table 1

Completeness of Required Fields (Composite Figures, n = 412)

Note. Composite figures. Completeness is the share of reports with a valid, non-blank value.

Standard 2: The Reporting Deadline

The state's rule requires laboratories to report carbapenem-resistant isolates within one business day of the result. Measured from the date the final susceptibility result was released to the date the state received the report, 83% of electronic reports met the deadline, compared with 46% of faxed reports. Electronic reporting's advantage in timeliness has been documented elsewhere; in one comparison, electronic laboratory reports arrived about eight days earlier than spontaneous paper reports (Overhage et al., 2008).

Standard 3: The Containment Window

The healthcare-associated infections team aims to start screening of contacts within three days of learning of a confirmed case, but the clock that matters for spread runs from specimen collection. Measured that way, the median interval from collection to state notification of confirmation was seven days, and the longest delays involved isolates shipped to the state laboratory for mechanism testing. Most of the distance from the containment goal lies before the state is notified, a finding consistent with the evaluation completed in Week 4.

Validity

Completeness measures whether a field is filled; validity measures whether the value is right. A sample of 60 reports was checked against facility records. Facility type was recorded incorrectly in 9, usually listing an acute care hospital when the patient had been transferred from a nursing home. Specimen source was mislabeled in 4, most often a rectal screening swab recorded as stool from a clinical specimen. Errors of the first kind send the containment call to the wrong facility; errors of the second can make a screening result look like an infection. The updated surveillance evaluation guidelines treat data quality as including both completeness and validity (German et al., 2001), and this appraisal shows why both are needed.

Why the Standards Differ

The three standards measure different things, and a report can meet one while missing another. A laboratory can send every required field on time and still deliver a report too late for containment, because the clock that matters for spread starts at specimen collection, before the laboratory has a result. Conversely, a report can arrive within the containment window but lack the facility type needed to direct the response. Reporting each standard separately keeps these failures distinct. It also assigns responsibility clearly: field completeness belongs mainly to laboratory interfaces, the reporting deadline to laboratory workflows and the containment window to the whole chain, including isolate shipment.

How the Appraisal Would Change Practice

The appraisal's results should be shared with each reporting laboratory in a short scorecard showing its completeness for each required field, its share of reports meeting the deadline and its median time from collection to notification for confirmed cases. Laboratories rarely see these measures, and feedback has been described as one of the more effective interventions for improving electronic surveillance data (Dixon et al., 2013). The program should repeat the appraisal every six months so laboratories can see whether changes have helped. Over time, the scorecards would also show the program which laboratories need technical assistance and which have solved their problems, letting scarce staff time go where it does the most good. A laboratory whose completeness for facility type rises from half to nine in ten after a small interface change offers a model the program can share with others using the same software. Publishing the aggregate results, without naming laboratories, would also let partners see the state of the system as a whole. Laboratories that see their own position against the state average often move without further prompting, which makes the scorecard one of the cheapest improvements available to the program. It requires only data the program already holds.

What the Appraisal Recommends

Make facility type and carbapenemase testing status required fields in the electronic reporting interface, so messages without them are flagged at receipt. Move remaining fax reporters to electronic reporting, starting with the laboratories serving nursing homes. Measure timeliness from specimen collection, not from receipt, in routine reports. Repeat the validity check annually on a small sample, because errors that a completeness count cannot see can send a response to the wrong place.

References

Dixon, B. E., Siegel, J. A., Oemig, T. V., & Grannis, S. J. (2013). Electronic health information quality challenges and interventions to improve public health surveillance data and practice. Public Health Reports, 128(6), 546-553. https://doi.org/10.1177/003335491312800614

German, R. R., Lee, L. M., Horan, J. M., Milstein, R. L., Pertowski, C. A., & Waller, M. N. (2001). Updated guidelines for evaluating public health surveillance systems: Recommendations from the Guidelines Working Group. MMWR Recommendations and Reports, 50(RR-13), 1-35.

Overhage, J. M., Grannis, S., & McDonald, C. J. (2008). A comparison of the completeness and timeliness of automated electronic laboratory reporting and spontaneous reporting of notifiable conditions. American Journal of Public Health, 98(2), 344-350. https://doi.org/10.2105/AJPH.2006.092700