I'm trying to understand the calculation of the t-statistic in a linear regression when you are testing against a specific value, i.e. using the formula $$t=\frac{\hat{\beta}-\beta}{\text{S.E.}(\hat{\beta})}.$$ t-statistic states that "$\text{S.E.}(\hat{\beta})$ is the standard error of the estimator ${\hat {\beta }}$ for $\beta$."
I understand how this works for $\beta=0$ (using $\sigma^2(X^TX)^{-1})$ but I'm not sure how to generalise the calculation of $\text{S.E.}(\hat{\beta})$.