With regards to hierarchical models I often see these referred to as groups borrowing information from each other e.g.
It will be seen that the hierarchical model posterior estimates for one school borrows information from other schools
I was hoping someone could explain where this interpretation comes from. As example lets say we fit the following model:
$$ \begin{align} x_{ij} &\sim N(\mu_i, \sigma) \\ \mu_i &\sim N(\mu', \tau) \\ \end{align} $$
Where $\mu', \tau, \sigma$ are all parameters that we need to provide priors for. My basic understanding is that:
- If we put a strongly informative prior on $\tau \to \infty$ then we are basically saying that the $\mu_i$ are independent e.g. there is no "borrowing"
- If we put a strongly informative prior on $\tau \to 0$ then we are basically saying that the $\mu_i = \mu$ e.g. they should all be regarded as coming from the same group with a singular mean.
However what does it mean if we put a weakly informative or non-informative prior on $\tau$ ? Does the parameter estimate of $\mu_i$ meaningfully impact the estimate of $\mu_j$? Or is this equivalent to modelling $\mu_i$ independently and then fitting $\mu'$ and $\tau$ to the $\mu_i$ as if they were fixed ? That is, if we have a weakly informative prior is there any "borrowing" and if so how strong is it and what impact does this practically have ?
