Perhaps the more general context of an affine space can be helpful. When we discuss $\mathbb R^n$, we typically think of the origin as a special point. We think of vectors as ordered tuples of real numbers. The we can add on the image of an arrow that has its tail at the special point (origin) and its tip at the point corresponding to the ordered tuple.
But in physics, we tend to think of vectors as arrows that go between any two points, and consider two such entities as equal if they are just translations of one another. So the vector tailed at $(1,1)$ with tip at $(4,2)$ is equal to the one tailed at $(3,-1)$ and tipped at $(6,0)$. We go from the first to the second by translating the plane by isometry $(x,y)\rightarrow (x+3,y+1)$.
Notice that the translation $(x,y)\rightarrow (x+3,y+1)$ is technically not a transformation of ${\mathbb R^2}$ as a vector space because it does not preserve the identity for addition, i.e., it moves the origin. The transformation group is the general linear group $GL(2,{\mathbb R})$, which can be represented as the 2x2 matrices with non-zero determinant.
So what is going on? The collection of arrows that are vectors for physicists form what is called an affine space, as opposed to a vector space. The geometer Marcel Berger said "An affine space is nothing more than a vector space whose origin we try to forget about, by adding translations to the linear maps." The definition of an affine space is kind of tricky. nLab might give you some ideas. One characterization found there is that an affine space is simply a vector space, but with different morphisms; an affine linear map is a function that is the difference between a linear map and a constant function. There are many ideas on the nLab site, with varying degrees of abstraction.
In Newtonian physics, we inhabit a "big cube" that is three-dimensional. Sometimes we over-step and think of it as the vector space ${\mathbb R^3}$. But in Newtonian physics, there is no distinguished point like the origin, e.g., the "me reference frame" which puts me at the center of the universe. So the big cube of Newtonian physics is really an affine space, not a vector space. The symmetry group of Newtonian physics is larger than the symmetry group for an associated vector space.
https://matheducators.stackexchange.com/questions/21048/big-list-of-interesting-abstract-vector-spaces
– user52817 Feb 19 '24 at 14:08