Intuition for relative error for vectors

Question

I'm trying to understand the notion of relative error for vectors in $\mathbb{R}^n$, but it's not "clicking" somehow.

$$\operatorname{\varepsilon-rel}(x_\text{approx}, x) = \frac{||x_\text{approx} - x ||}{||x||}$$

The intuition I have for the relative error between two positive real numbers is that it's the approximation of something symmetric $$\operatorname{\varepsilon-rel}(x_\text{approx}, x) \approx \log(x_\text{approx})-\log(x)$$ which I can think about as a (signed) distance metric.

Here's what I can say about the vector case:

1. It's invariant under rotation
2. It's invariant under uniform scaling
3. If $x$ and $x_\text{approx}$ are collinear, say $x_\text{approx} = \alpha x$, then you recover the relative error

But 3. kinda breaks down when $\alpha < 0$. The univariate relative error is $\infty$ if the signs differ, the multivariate is not.

4. If the error $x - x_\text{approx}$ is orthogonal to $x$ then $\operatorname{\varepsilon-rel}$ is the $\sin$ of the angle between $x$ and $x_\mathrm{approx}$.

Your thoughts would be much appreciated.

Answer 1

You are overthinking relative error in one-dimension, and I expect that is the source of your confusion.

If I measure the length of an ant, and I am off by 1mm, its a big deal. However, if I were instead to measure the distance from the earth to the moon, and I was off by the same amount, for all practical purposes my measurement is perfect, relative to that distance 1mm is so small it can effectively be considered zero. This is what relative error measures. In both instances the absolute error is the same, however the relative error in the first measurement is much larger than in the second. The extent to which this holds in other vector spaces depends on whether the induced metric for the norm you are using can be fairly said to measure how "closely" one vector approximates another vector. (the induced metric is the value $\Vert v-w \Vert$). In situations where it does relative error represents/measures/means the same thing it does in 1 dimension. In situations where it doesn't the relative error is obviously still well-defined but serves no purpose. It may pop up but in those instances it is no longer referred to as relative error. Sometimes other norms besides the Euclidean Norm will be more suitable for capturing this idea. The importance of the phrase "it depends on the context" in mathematics cannot be overstated.

Answer 2

Let $x \in \mathbb{R}^{n}\setminus \{0\}$ be a point and $\mathrm{\delta}x \in \mathbb{R}^{n}$ be a tangent vector at $x$.

The relative error $||\delta x||_x := \frac{||\delta x||}{||x||}$ defines a Riemannian metric on $\mathbb{R}^{n}\setminus \{0\}$.

Case $n = 1$

Let $0 < x_1 < x_2 \in \mathbb{R}$. The shortest path is given by $$x(t) = x_1 + (x_2-x_1) t.$$

Its length is $$ \begin{aligned} \int_0^1 || \dot{x}(t) ||_{x(t)} \mathrm{d}t &= \int_0^1 \frac{x_2 - x_1}{x_1 + (x_2 - x_1) t} \mathrm{d}t \\ &=\log(x_2) - \log(x_1). \end{aligned}$$

It follows that the distance between two points is given by $$ d(x_1, x_2) = |\log(|x_1|) - \log(|x_2|)|$$ if they have the same sign and $\infty$ otherwise. This is a symmetric, non-infinitesimal version of the relative error.

Case n > 1

In this case, the space $\mathbb{R}^{n}\setminus \{0\}$ is the product of the sphere $S^{n-1}$ (with the Euclidean metric) and $\mathbb{R}^+$ (with relative error metric defined above).

For example, when $n=2$, the Riemannian metric for relative error is given by $$ g = \frac{1}{x^2 + y^2} \left( \mathrm{d}x \otimes \mathrm{d}x+ \mathrm{d}y \otimes \mathrm{d}y \right).$$

Or equivalently in polar coordinates: $$ g = \frac{1}{r^2} \mathrm{d}r \otimes \mathrm{d}r+ \mathrm{d}\theta \otimes \mathrm{d}\theta$$ which is the product metric of the relative error on $\mathbb{R}^+$ and arc length on the circle $S^1$.

It follows that the distance between two points is given by $$ d(p, q) = \sqrt{\left( \log ||p|| - \log ||q|| \right)^2 + \left[ \cos^{-1}\left( \frac{p \cdot q}{||p|| \cdot ||q||} \right) \right]^2}.$$ This is a symmetric, non-infinitesimal version of the relative error between two vectors.