HLTH 8202E Week 11 Survey Instrument Defense Example

Defending the Home Health Aide Heat Survey as a Chain of Decisions, From Three Constructs to a Two-List Frame, Each Link Set Against the Alternative a Skeptical Reviewer Would Prefer

Student Name

Walden University

HLTH 8202E: Survey Research Methods

Instructor Name

Month Day, Year

Defending the Home Health Aide Heat Survey as a Chain of Decisions, From Three Constructs to a Two-List Frame, Each Link Set Against the Alternative a Skeptical Reviewer Would Prefer

The Chain

The finished questionnaire on heat in clients' homes, answered by aides, is the product of decisions that constrain each other. The constructs fix which items are needed. The items fix what options make sense. The options, together with the sensitivity of some questions, fix the order. And the population fixes the mode and the frame. This paper defends the instrument link by link. At each link it names the alternative a skeptical reviewer would prefer, explains why that alternative lost and cites the evidence behind the choice: a pretest finding, a named bias or the needs of the analysis.

Decision Log

Table 1

Major Decisions, the Alternative Each Rejected and the Reason

Each row is defended in a section below in the order the chain runs.

Constructs Before Items

A reviewer might ask why the survey does not use a single scale of heat at work, which would be shorter and easier to report. The answer is the analysis. The question member agencies most need answered is whether giving aides control over cooling reduces strain. That question cannot be asked unless exposure, control and strain are measured separately. Groves and colleagues describe validity as the correspondence between a construct and its measure (Groves et al., 2009); a single scale would correspond to a blend of three ideas.

Items That Report Rather Than Diagnose

A reviewer might prefer to ask aides whether they had heat exhaustion, since that is the outcome of clinical interest. The pretest showed why not: aides read the term in different ways, from any discomfort to an emergency. Tourangeau and colleagues locate such problems at comprehension and judgment (Tourangeau et al., 2000). Asking about specific symptoms, which aides can recall, lets the analysis group them by severity consistently. The same logic shaped the cooling item. An early draft asked whether aides took breaks in hot homes, and pretest interviews showed that aides paid by the visit did not count a pause as a break. Describing the actions themselves, stepping outside or sitting in a car with air conditioning, produced answers that meant the same thing for every aide.

Options That Leave Room for the Truth

A reviewer might argue that a no-visits option complicates the scale. It does, slightly, but without it aides who did not work last summer would choose "none of my visits" and appear unexposed, which would bias the exposure estimate downward. The option also lets those aides skip the rest of the exposure section quickly. Replacing agree-disagree statements with item-specific questions follows evidence that agreement formats invite acquiescence, especially among respondents who are tired or less engaged (Krosnick, 1991).

Order Set by Sensitivity

A reviewer might want questions about training and employer support at the start, since they matter most to agencies. They come near the end instead. Aides may worry that answers about their employer will be seen, and Tourangeau and Yan (2007) reviewed evidence that sensitive questions raise nonresponse and lead respondents to edit their answers. Placing exposure and symptom items first lets aides describe their experience before they are asked to evaluate anyone, and it builds trust before the more sensitive items.

Mode and Frame Set by the Population

A reviewer might prefer a mail survey, which is familiar and reaches people without smartphones. But aides in a high-turnover field change addresses often, and agencies and the consumer-directed program hold phone numbers more reliably than addresses. A text invitation with a web link, followed by telephone calls to nonrespondents, follows the logic of tailored design, which matches contact methods to the population and uses several contacts to build response (Dillman et al., 2014). The frame combines agency rosters and the consumer-directed payroll file because rosters alone would miss aides employed through the program; aides on both lists are weighted for their two chances of selection (Lohr & Rao, 2006).

Languages Before Launch

A reviewer might accept an English-only survey with a translated version added later, on the grounds that most aides read English. The decision was to translate before fielding and to pretest each language in its own right. Aides who prefer another language are a large part of the workforce in many states, and they may also be the aides with the least control over their working conditions. A survey that reached them later, or in a translation never tested, would measure their experience less accurately than everyone else's. The bilingual panel that reviewed translations included aides, so that terms like "comfortably" were rendered as aides would say them.

Weights and Reported Rates

A reviewer might ask why the survey reports a weighted estimate and a named response rate rather than a simple percentage of respondents. The answer is that the sample was stratified and drawn from two overlapping lists, so unweighted results would overrepresent some groups. Nonresponse adjustment uses variables available for everyone on the frame. Reporting the response rate by a named standard definition lets readers compare it with other surveys, since rates can be calculated in many ways and an unlabeled figure invites misreading.

What a Reviewer Might Still Reject

Some decisions remain open to reasonable disagreement. A reviewer might argue that the three-month recall window is still too long for symptom frequency, and that a diary study over two weeks of a heat wave would be more accurate. That is true, and a diary study would be a good next step. But it would capture one heat wave rather than a season and would demand far more of each aide. The survey chooses breadth across the summer and accepts some recall error as the price.

What the Instrument Cannot Do

A defense that claimed too much would be weaker. The survey measures perceived heat, not temperature. It measures symptoms, not diagnoses. It is cross-sectional, so an association between control and lower strain does not prove that control reduces strain. It relies on recall over a three-month window, which reduces but does not remove memory error (Sudman & Bradburn, 1973). And its frame leaves out aides hired directly by families and those whose agencies declined to share rosters, who may face the hardest conditions. These limits are stated in the methods section and should be stated wherever the results are reported.

Conclusion

Each decision in the instrument follows from the one before it, and each rejected alternative had a reason. The pretest supplied evidence for the item wording; published research on response behavior supplied evidence for the options and order; and the population supplied the reasons for the mode and frame. The result is a survey that measures what aides can report, says plainly what it cannot measure, and can answer the question the association's members care about most.

References

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.

Groves, R. M., Fowler, F. J., Jr., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd ed.). Wiley.

Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213-236. https://doi.org/10.1002/acp.2350050305

Lohr, S., & Rao, J. N. K. (2006). Estimation in multiple-frame surveys. Journal of the American Statistical Association, 101(475), 1019-1030. https://doi.org/10.1198/016214506000000195

Sudman, S., & Bradburn, N. M. (1973). Effects of time and memory factors on response in surveys. Journal of the American Statistical Association, 68(344), 805-815. https://doi.org/10.1080/01621459.1973.10481428

Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press.

Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859-883. https://doi.org/10.1037/0033-2909.133.5.859