4

I am using Catch v2.13.1

What is the correct way to compare float values. I thought the below would fail, but both pass.

REQUIRE(1147332687.7189338 == Approx(1147332688.4281545).margin(0.0001));
REQUIRE(1147332687.7189338 == Approx(1147332688.4281545));

However this fails as expected

REQUIRE(abs(1147332687.7189338 - 1147332688.4281545) <= Approx(0).margin(0.0001));

I don't understand why the first two statements wouldn't work

Marek R
  • 27,988
  • 5
  • 42
  • 123
Inquisitor
  • 43
  • 1
  • 4

2 Answers2

3

There are a couple of things to consider in the posted example.

REQUIRE(1147332687.7189338 == Approx(1147332688.4281545));

This will pass, "unexpectedly". The reason can be found in the documentation (assertions - floating point comparisons).

Approx is constructed with defaults that should cover most simple cases. For the more complex cases, Approx provides 3 customization points:

  • epsilon - epsilon serves to set the coefficient by which a result can differ from Approx's value before it is rejected. By default set to std::numeric_limits<float>::epsilon()*100.
  • [...]

In the posted example, the two numbers differ by a cofficient near 6.2e-10, while the default is (given a 32-bit float) near 1.2e-5.

The following test wouldn't pass.

CHECK( a == Approx(b).epsilon(1e-12) );

The other tests involve margin, which is described in the documentation as

  • margin - margin serves to set the the absolute value by which a result can differ from Approx's value before it is rejected. By default set to 0.0.

The caveat, though, can be found in issue#1507.

This is because of the default value for epsilon in the Approx class and the fact that Approx::equalityComparisonImpl will pass if the value is in the range of the margin OR the epsilon values.

So, this test wouldn't pass:

CHECK( a == Approx(b).margin(0.0001).epsilon(1e-12) );

Note that this "issue" seems to be marked as resolved - not a bug:

So, I don't think this is a bug.

The reason for this is that it is very hard (well, impossible), to determine the user's intent, thus it is better to assume that the user has set up both checks correctly -- after all, if the user does not want a relative comparison, they can always set the epsilon to zero. In fact, I think that using both tolerances and taking the most forgiving one is the superior option to implement things like the Approx Matcher (#1499).

Bob__
  • 10,913
  • 3
  • 25
  • 37
  • 1
    Thank you, I don't understand why margin exists if it has incorrect behavior depending on what epsilon is set to. – Inquisitor Sep 30 '20 at 21:58
  • @Inquisitor Yeah, that's a debatable design choice, IMHO. Robust floating point comparison is kind of an esoteric art, though, see e.g. https://stackoverflow.com/questions/5595425/what-is-the-best-way-to-compare-floats-for-almost-equality-in-python and the link there. – Bob__ Sep 30 '20 at 22:15
0

I think I found out why this happens, but I don't know how to make it work in Catch.

This:

#include <iostream>
#include <limits>
#include <iomanip>

int main()
{
    double a = 1147332687.7189338;
    double b = 1147332688.4281545;
    float c = 1147332687.7189338;
    float d = 1147332688.4281545;

    std::cout << std::setprecision(std::numeric_limits<decltype(a)>::max_digits10) << a << std::endl;
    std::cout << std::setprecision(std::numeric_limits<decltype(b)>::max_digits10) << b << std::endl;
    std::cout << std::setprecision(std::numeric_limits<decltype(c)>::max_digits10) << c << std::endl;
    std::cout << std::setprecision(std::numeric_limits<decltype(d)>::max_digits10) << d << std::endl;
}

Outputs this:

1147332687.7189338
1147332688.4281545
1.14733274e+09
1.14733274e+09

So clearly floats are not good enough to distinguish these numbers but doubles are. It seems that Catch uses floats internally; the test passes even when I try to force it to use doubles. But maybe there is something I missed?

TEST_CASE("Test1")
{
    double d1 = 1147332687.7189338;
    double d2 = 1147332688.4281545;
    REQUIRE(d1 == Approx(d2));
}

So no fix, but at least you now know why you get those strange results.

Frodyne
  • 2,473
  • 4
  • 14