1

In a book Intepretable Machine Learning by Christoph Molnar there is a following passage:

https://christophm.github.io/interpretable-ml-book/limo.html#example

I don’t understand how can we talk about a standard error of the estimated coefficient. I understand standard error as standard deviation of a given sample o values. However, when we are talking about a given weight of a linear model there is no sample, or rather the sample consists of just one value - the estimated weight, so the standard error would be 0. What am I missing?

Link to the book: https://christophm.github.io/interpretable-ml-book/limo.html#example

czypsu
  • 111

2 Answers2

2

The standard error comes from a thought experiment. Say we drew not one but many samples of $N$ persons from a population, and in each of these samples we estimated the same regression model. We could estimate the parameter or weight of a given explanatory/right-hand-side/indepedent/$x$-variable in each sample and store those. This (hypothetical) collection of weights is the sampling distribution for that parameter, and the standard deviation of that sampling distribution is what we call the standard error.

Maarten Buis
  • 21,005
0

To complete the answer by @MaartenBuis,

Standard deviation is the measure of variability of a sample with individual values (observations).

Standard error is the measure of variability of a sample statistic (for example : mean, proportion, odds ratio, incidence rate or regression coefficient)

So, we can talk about a standard error of an estimated coefficient. Say $S_0$ is the sample you used to get the estimated coefficient. Then you take multiple such samples $S_1,S_2,...,S_n$ and generate an estimated coefficient in each sample. So here ultimately we have a sample made of estimated coefficients (each observation in the sample now is an estimated coefficient). And the standard deviation of this sampling distribution is called standard error.

Happy Cretine
  • 669