How can we incorporate an offset into a software library which performs least squares without offset

Question

Disclaimer: this is not homework, just preparation for an exam I'll be taking

How do I incorporate an offset term into a software library that performs least squares (without offset)?

Let me give a simple example which I'd like for the answerer to base their answer on:

Consider a dataset $(x_1,y_1),...,(x_m,y_m)\in R \times R$, i.e., both $x_i$ and $y_i$ are real-valued scalars for all $i$. Suppose we wish to fit a linear+offset model, $$\hat{y} = ax + b$$ Suppose we have a software library that performs least squares (without offset). Explain how we can use that software library to fit the linear+offset model.

Now, by my understanding, we have that the software library performs linear regression on $X\in R^m$ and $y\in R^m$. In order to incorporate an offset, we should add a column of "ones" such that $X$ is substituted by $(X,\mathbf{1})$ where $\mathbf{1}\in R^m$ is a vector with each entry equal to $1$. We also should substitute $a\in R$ by $(a,b)$ where $b\in R$. Now, without the offset, the normal equations for computing $a$ was $$a= (X^TX)^{-1}X^Ty$$ and with the above substitutions the software library calculates $$(X^TX)a + X^T\mathbf{1}b = X^Ty\\ \mathbf{1}^TXa + mb = \mathbf{1}^Ty$$

Is my understanding correct?

Answer 1

You can just subtract the offset $b$ from $y$ and proceed as normal using OLS. So your regression coefficients would simply be $$ (X'X)^{-1}X'(y-b) $$ Nothing related to the offset needs to be added to the design matrix ($X$). For nonlinear models, the story is different, but for OLS, it's simple.

You can see this by considering the least squares criterion that is minimized: $$ \sum_i{(y_i-(aX_i + b_i))^2} = \sum_i{((y_i-b_i) - aX_i))^2} $$ So performing OLS with an offset is just the same as performing OLS after subtracting the offset from the outcome.