PHLT 8500 Week 9 Model Comparison Example

Does a Curve Earn Its Extra Term? Comparing a Linear and a Spline Specification of Log Well Arsenic in the Hemoglobin A1c Model, and Choosing Between Them by the Model's Purpose

Student Name

Walden University

PHLT 8500: Advanced Biostatistics

Instructor Name

Month Day, Year

Does a Curve Earn Its Extra Term? Comparing a Linear and a Spline Specification of Log Well Arsenic in the Hemoglobin A1c Model, and Choosing Between Them by the Model's Purpose

Purpose Before Measures

Model comparison needs a criterion, and the criterion comes from what the model is for. The Week 2 to Week 4 model of hemoglobin A1c in the composite private-well cohort exists to estimate how A1c differs with well arsenic, not to predict A1c for individuals. A specification that predicts slightly better but leaves the arsenic estimate unchanged offers little for that purpose; a specification that changes the arsenic estimate matters even if fit statistics barely move. With that rule stated, this paper compares two specifications that differ in one respect: how log arsenic enters the model.

The Two Candidates

Model A, carried forward from Week 4, enters log base 2 of well arsenic as a single linear term, assuming each doubling of arsenic carries the same difference in A1c across the whole range. Model B replaces that term with a restricted cubic spline with three knots, at the 10th, 50th and 90th percentiles of log arsenic, which adds one term and allows the association to bend. Both models carry the identical covariate set from earlier weeks. The question is whether the bend is real enough to justify the extra term.

Nested, So Testable

Model A is a special case of Model B: setting the spline's nonlinear coefficient to zero returns the linear model. Because the models are nested, a likelihood ratio test compares them directly. The test statistic is 2.6 on 1 degree of freedom, p = .11. The data provide weak evidence at most that the association departs from linearity on the log scale.

Table 1

Fit Statistics for the Two Specifications (Composite Figures, n = 1,612)

Note. Composite figures. Lower information criteria indicate better balance of fit and complexity.

Two Criteria Disagree

The Akaike information criterion is slightly lower for Model B, by 0.6, while the Bayesian information criterion is lower for Model A, by 4.8. The disagreement is expected. Both criteria reward fit and penalize parameters, but BIC's penalty grows with sample size and is much heavier at 1,612 participants, so BIC leans toward simpler models. Burnham and Anderson (2004) explain that the two criteria answer different questions: AIC targets the model expected to predict new data best, while BIC targets the model most likely to be the true one among the candidates, under assumptions rarely met in practice. They also suggest that an AIC difference under about 2 means both models have substantial support, which describes this comparison. On fit alone, neither model is clearly better.

Tracking the Exposure

Because fit statistics do not decide the matter, the purpose rule does. What does each model say about arsenic?

Model B suggests a slightly steeper rise at low concentrations and a flatter one at high concentrations, a pattern sometimes seen with environmental exposures. But the differences between the models are small relative to the confidence intervals, which are about plus or minus 0.035 for the interquartile contrast, and the summary contrast across the middle half of wells, 0.048 versus 0.052, is nearly identical. The flattening at high concentrations rests on few wells: only about 140 participants have wells above 40 micrograms per liter.

Table 2

Predicted A1c Differences Under Each Specification (Composite Figures)

Note. Composite figures; differences are A1c percentage points.

One Kept, One Dropped

Model A is kept as the primary specification. It is simpler to report, its single coefficient summarizes the association in a form readers can use, and the spline does not change the conclusion. Model B is reported as a sensitivity analysis, with a figure of its curve, so readers can see that the linear form is a reasonable summary. Greenland (1989) cautions that model choice can affect effect estimates and their intervals, and reporting both specifications shows that, here, it does not. Westreich and Greenland (2013) recommend that tables focus on the exposure estimate, and both tables in this paper do.

Why Not Decide by the Test Alone

The likelihood ratio test's p-value of .11 could be read as a verdict: not significant, so keep the linear model. That reading would reach the right decision here for the wrong reason. A nonsignificant test does not show that the association is linear; it shows that the data cannot clearly distinguish the two shapes. With a larger cohort, the same small bend might reach significance without changing the arsenic estimate in any way that matters. Conversely, a significant test would not by itself justify the spline if the two models gave the same answer to the question at hand. The test, the information criteria and the comparison of estimates are all evidence, and the purpose rule decides how to weigh them.

What Would Change the Decision

If the spline had shifted the interquartile estimate substantially, or if a policy question focused on low concentrations, where the curve is steeper, Model B would be preferred despite its extra term. A future analysis of whether health effects occur below the 10-microgram federal standard would need the flexible form, because the linear model imposes a shape on exactly the range that question concerns.

Summary

The linear and spline specifications fit almost equally well: the likelihood ratio test is not significant, AIC slightly favors the spline and BIC favors the linear model. Judged by the model's purpose, estimating the arsenic association, the two give nearly the same answer. The linear model is kept and the spline reported as a sensitivity check, with a note that questions about low concentrations would call for the spline.

References

Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods & Research, 33(2), 261-304. https://doi.org/10.1177/0049124104268644

Greenland, S. (1989). Modeling and variable selection in epidemiologic analysis. American Journal of Public Health, 79(3), 340-349. https://doi.org/10.2105/AJPH.79.3.340

Westreich, D., & Greenland, S. (2013). The Table 2 fallacy: Presenting and interpreting confounder and modifier coefficients. American Journal of Epidemiology, 177(4), 292-298. https://doi.org/10.1093/aje/kws412