A homogeneous transformation matrix $H$ is often used as a matrix to perform transformations from one frame to another frame, expressed in the former frame. The translation vector thus includes [x,y(,z)] coordinates of the latter frame expressed in the former. Perhaps that this already answers your question, but below is a more elaborate explanation.
The transformation matrix contains information about both rotation and translation and belongs to the special Eucledian group $SE(n)$ in $n$-D. It consists of a rotation matrix $R$ and translation vector $r$. If we permit no shear, the rotation matrix contains only information about the rotation and belongs to the orthonormal group $SO(n)$. We have:
$$H=\begin{bmatrix} R & r \\ \bar{0} & 1 \end{bmatrix}$$
Let's define $H^a_b$ the transformation matrix that expresses coordinate frame $\Phi_b$ in $\Phi_a$, expressed in $\Phi_a$. $\Phi_a$ can be your origin, but it can also be an other frame.
You can use the transformation matrix to express a point $p=[p_x\ p_y]^\top$ (vectors) in another frame:
$$P_a = H^a_b\,P_b$$
$$P_b = H^b_c\,P_c$$
with
$$P = \begin{bmatrix} p \\ 1 \end{bmatrix}$$
The best part is that you can stack them as follows:
$$P_a = H^a_b H^b_c\,P_c = H^a_c\,P_c $$
Here a small 2 D example. Consider a frame $\Phi_b$ translated $[3\ 2]^\top$ and rotated $90^\circ$ degrees with respect to $\Phi_a$.
$$H^a_b = \begin{bmatrix}\cos(90^\circ) & -\sin(90^\circ) & 3 \\ \sin(90^\circ) & \cos(90^\circ) & 2 \\ 0 & 0 & 1 \end{bmatrix}=\begin{bmatrix}0 & -1 & 3 \\ 1 & 0 & 2 \\ 0 & 0 & 1 \end{bmatrix}$$
A point $p_b=[3\ 4]^\top$ expressed in frame $\Phi_b$ is
$$\begin{bmatrix}p_{a,x} \\ p_{a,y} \\ 1 \end{bmatrix} = \begin{bmatrix}0 & -1 & 3 \\ 1 & 0 & 2 \\ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix}3 \\ 4 \\ 1 \end{bmatrix}=\begin{bmatrix}-1 \\5 \\1 \end{bmatrix} \to p_a = \begin{bmatrix}-1\\5\end{bmatrix}$$
Try to make a drawing to improve your understanding.
