Robustness in general refers to a statistic's insensitivity to deviations from its underlying assumptions (Huber and Ronchetti, 2009).
Robust statistics are insensitive to deviations from their underlying assumptions and outliers. Such methods are useful it is not possible to detect and remove outliers or to appropriately test the assumptions required by a given statistic. A robust statistic is meant to achieve three goals:
- efficiency - it should have an optimal or nearly optimal efficiency as the assumed model
- stability - small deviations from the assumptions should have only a small influence on performance
- breakdown - larger deviations from the assumptions should not lead to a complete failure
Examples of robust statistics are median regression as estimation technique, or Huber-White standard errors for statistical inference. Note that "robust" is not equivalent to "better". Robustness is always based on compromise as it sacrifices efficiency to ensure against larger deviations from the assumptions from the model (Anscombe, 1960).
For further reading see
- Huber, P.J. and Ronchetti, E.M. (2009) "Robust Statistics", 2nd Edition, Wiley Series in Probability and Statistics, John Wiley & Sons, Inc., New Jersey
- Anscombe, F.J. (1960) "Rejection of Outliers", Technometrics, Vol. 2, pp. 123-147
