Robustness is Parameter-Specific: A Comment on Rasch and Guiard's Robustness Study

Rasch and Guiard (2004) make the case that nonparametric procedures can often be set aside, since parametric tests hold up well even when data depart from normality. The reasoning deserves a second look. Treating robustness as a single property, measured against a single kind of violation, misses how many ways data can break the assumptions a test rests on. Non-normality is one such violation. It is not the only one, and a decision to prefer parametric methods built on that one result rests on narrower ground than it first appears.

Consider the t-test, the usual exhibit when people argue that parametric methods tolerate messy data. Against departures from normality it behaves almost as if nothing were wrong; its error rates barely move. That reputation is earned, but it is specific. The same test reacts very differently when a different assumption fails. Independence of observations is one case in point. When observations are correlated, as happens with autocorrelated time series or clustered measurements, the t-test loses the protection it enjoys against non-normality. Here it is sensitive, and sharply so.

The practical lesson is that robustness attaches to particular assumptions rather than to a test as a whole. A method can shrug off one kind of trouble while being wrecked by another. So the finding that certain tests survive normality violations cannot, by itself, justify discarding distribution-free alternatives. That conclusion skips over the assumptions where parametric tests are fragile, and it does not weigh how nonparametric tests perform under those same conditions.

What this points toward is more study, not a verdict. We need work that examines violations one assumption at a time, independence among them, and that puts parametric and nonparametric procedures through the same conditions side by side. Until that comparison exists, writing off nonparametric tests on the strength of their rivals' resistance to non-normality is premature.