Skip to content

Add comparison macros for floating point numbers (ck_assert_floating_*) #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Dec 10, 2016

Conversation

andr-dots
Copy link
Contributor

@andr-dots andr-dots commented Nov 6, 2016

Those macros could help testing geometry functions with specified precision.

@andr-dots
Copy link
Contributor Author

I'm a newbye to github. Could somebody explain why does AppVeyor fails and how to debug this?

@prusnak
Copy link
Contributor

prusnak commented Nov 6, 2016

C:\projects\check\tests\check_check_sub.c(595) : error C2124: divide or mod by zero

It seems that MSVC has build-time check for this.

@andr-dots andr-dots force-pushed the master branch 2 times, most recently from 76c3ad5 to b9d11c7 Compare November 6, 2016 14:18
@andr-dots
Copy link
Contributor Author

Need some help again. I've figured out how to open logs for ms builds. With nan I'll deal somehow on different compilers. But logs for platform=ms are unclear for me. Can I get tests log somehow with asserts text?

@andr-dots
Copy link
Contributor Author

I've removed nan, finite and infinite asserts until get managed to check them on any compiler (ignoring nan's sign).

@brarcher
Copy link
Contributor

brarcher commented Nov 6, 2016

Jenkins: ok to test
^ Will start the last bit of the tests

*
* @since 0.10.1
*/
#define ck_assert_floating_eq(X, Y) _ck_assert_floating(X, ==, Y)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may not want to offer a way to use the == operator on two floating point numbers. Because floating point math is not associative and may follow different rounding models, the outcome of a computation may not be exactly what one expects. The ck_assert_floating_eq_prec() call below is a much better option. I would request that this and the ck_assert_floating_ne() methods be removed.

@@ -104,6 +104,22 @@ static master_test_t master_tests[] = {
{ "Simple Tests", CK_FAILURE, "Assertion 'x >= y' failed: x == 2, y == 3" },
{ "Simple Tests", CK_FAILURE, "Assertion '1%d >= 3%f' failed: 1%d == 0, 3%f == 1" },
{ "Simple Tests", CK_PASS, "Passed" },
/* Tests on floating macros */
{ "Simple Tests", CK_FAILURE, "Assertion 'x == y' failed: x == 1.100000, y == 1.200000" },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You will need to rebase against master. I added a change today which will add an entry to this table. Now one will need to include the name of the method under test. For example, the first entry will now become:

{ "Simple Tests", "test_ck_assert_floating_eq", CK_FAILURE, "Assertion 'x == y' failed: x == 1.100000, y == 1.200000" }

do { \
long double _ck_x = (X); \
long double _ck_y = (Y); \
unsigned char _prec = (PREC); \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PREC parameter currently assumes that one wants to compare the precision of decimal places assuming a power of 1. However, it may be likely that one is interested in comparing using other powers. For example, if I have two floating point numbers: 1e10 and 1.1e10 I may be interested in checking that they are within 1e8 of each other.

Consider changing this to allow one to pass in the maximum value that the two numbers can be apart and still be considered 'equal'.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Power of 10 is often used in calculations when assuming fix point calculations which are used in embedded systems. Through fixed point types are unable under C standard using float is much or less a good way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A'll consider adding another function for eps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation. I now have a better appreciation for where you are trying to go with this macro.

Given your use of the macro, having this to help with math which is expected to be an approximate for fixed point seems reasonable. I still believe that for the floating point macros to be complete a more general macro which allow one to pass the threshold should be provided. Can you provide both in your change?

}
END_TEST

START_TEST(test_ck_assert_floating_eq_prec)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When testing this method should data available in cfloat.h be used to determine the correctness? This will result in more extreme examples that check boundary conditions.

@brarcher
Copy link
Contributor

brarcher commented Nov 6, 2016

I'm OK considering this change. However, I warn that working with floating point values is hard and determining good and meaningful comparisons is non-trivial. Can you share your expected use case for these macros and the application you are considering using them to unit test?

@andr-dots
Copy link
Contributor Author

We use those tests in our commercial project so I cannot give full example. But it looks like this:
ck_assert_double_prec(&speed_integrated, &pos_delta, precision);
We use double instead of floating to rebase correctly in future.

#define _ck_assert_floating(X, OP, Y) do { \
long double _ck_x = (X); \
long double _ck_y = (Y); \
ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: %s == %.6Lf, %s == %.6Lf", #X" "#OP" "#Y, #X, _ck_x, #Y, _ck_y); \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having no expectation on the values being compared using the format specifier %.6Lf may be a little restrictive. For example, if X were 0.000000012 then the value of 0.000000 would be printed. However, the format %Le would print 1.200000e-08.

Would you be opposed to changing the format specifier to be more general?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about it. But tests should give human readable output. So in case of 0.001 and 1.0e-3, 0.001 is clear. But 1.0e-3 spends your time to do some calculations in your mind. In this case %Lg is universal solution. Just didn't checked what output could be on different compilers and system. In any case I'll make some changes in a week or two to tests making use regular expressions for output analysis (if don't mind) through nan and -nan problem appeared, so no problem, I'll add %Lg everywhere. I can contribute to open source only in free from work time so for now it wouldn't be fast.

@brarcher
Copy link
Contributor

brarcher commented Nov 8, 2016

Need some help again. I've figured out how to open logs for ms builds. With nan I'll deal somehow on different compilers. But logs for platform=ms are unclear for me. Can I get tests log somehow with asserts text?

Turns out that the Visual Studios build was never printing the output of the unit tests, even on failure. This is now resolved in master (this pull request). The Visual Studios build will now always emit the unit test output. Thanks for pointing this out.

@andr-dots andr-dots force-pushed the master branch 2 times, most recently from 1c1b0c1 to 0b62863 Compare November 8, 2016 21:11
long double _ck_y = (Y); \
int _ck_p = (PREC); \
int _prec = -_ck_p; \
ck_assert_msg(_prec <= 0, \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this assertion is to help protect the usage of this macro consider changing its check to something that relates directly to the user's input. That is,

ck_assert_msg((PREC) > 0, ...)

This will help a user make sense of the failure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User see custom message after condition and condition (PREC) > 0 won't be seen. How could it help?

@brarcher
Copy link
Contributor

brarcher commented Nov 9, 2016

There are a few failures in the unit tests. One of them I neglected to warn about, sorry about that. In addition to adding the name of the unit test to the master tests table, one will need to add the following call to the unit tests as well:

record_test_name(tcase_name());

The second failure is here:
/home/travis/build/libcheck/check/tests/check_check_master.c:379:F:Core Tests:test_check_failure_msgs:0: For test 60:Simple Tests:test_ck_assert_floating_eq_prec_min_prec Expected Assertion '-min_prec_minus_one <= 0' failed (prec): min_prec_minus_one == -1, got Assertion '-LDBL_MIN_10_EXP <= 0' failed (prec): LDBL_MIN_10_EXP == -4931

It appears that the test_ck_assert_floating_eq_prec_min_prec unit test failed (as expected) but it did not fail in the way it was expected to fail.

@andr-dots
Copy link
Contributor Author

My main problem is that
ctest -V -R check_check
goes toooo long because of sleeps. Is there simple any way to run tests by their name or something?

@brarcher
Copy link
Contributor

brarcher commented Nov 9, 2016

There is a way to disable all the timeout tests:

./configure --disable-timeout-tests

With that you will find that unit tests run very quickly.

@andr-dots andr-dots force-pushed the master branch 4 times, most recently from df857d5 to 2c445e8 Compare November 10, 2016 04:54
@andr-dots
Copy link
Contributor Author

andr-dots commented Nov 10, 2016

Well I've done some research. %Lg with default precision is confusing through 0.1 is not 0.1 because of digits at the end of number. So made macro CK_FLOATING_DIG that can be set by user to determine his own precision (for example DECIMAL_DIG for long double from C99). In documentation I wrote restriction on use of floating point numbers. And I cannot test bounds of precision because of different representations on x32 and x64 systems (or on different compilers/platforms, didn't dig for the reason). And because of big error coming from binary representation. But tests are useful in any case.

Macros for threshold and macros for eps would be added some later through for now I have no need of them in the project I work for. Eps would be next stage in experiment of improving precision of algorithms.

@andr-dots
Copy link
Contributor Author

andr-dots commented Nov 28, 2016

It's more complicated. For example, numbers 0.1f, 0.1 and 0.1l are not equal. So if someone would test float variable but second argument would be of double type (just number) the result would be erroneous.
That is because converting from one floating point type to another has loss of precision. For example if we convert float to double we would have the same 7 digits of precision. But double have 15 bits of precision. So they would show garbage.
http://stackoverflow.com/questions/3498192/c-convert-double-to-float-preserving-decimal-point-precision

So to prevent situations when user doesn't understand what is happening I added macros for every type.

Moreover if we would print a float number converted to long double type, than we would print different result than user expects to see.

@brarcher
Copy link
Contributor

For example, numbers 0.1f, 0.1 and 0.1l are not equal. So if someone would test float variable but second argument would be of double type (just number) the result would be erroneous.
That is because converting from one floating point type to another has loss of precision. For example if we convert float to double we would have the same 7 digits of precision. But double have 15 bits of precision. So they would show garbage.

This would apply for converting from a type with more precision (long double) to one with less precision (float). However, if one always converts to long double how would this result in an issue? Is there evidence that converting from float to long double results in 'garbage' (i.e. undefined behavior)?

@andr-dots
Copy link
Contributor Author

andr-dots commented Nov 28, 2016

Here is the example:
	printf("%g; %lg; %d\n", 0.1f, 0.1, 0.1f == 0.1);
// 0.1; 0.1; 0
// As shows printf they are equal, but...
	printf("%.15g; %.15lg; %d\n", 0.1f, 0.1, 0.1f == 0.1);
// 0.100000001490116; 0.1; 0
    volatile float x = 0.1f;
    volatile double y = 0.1;
    printf("%.15lg; %.15lg; %d\n", (double) x, y, ((double) x) == y);
// 0.100000001490116; 0.1; 0
// That was happening when converting float to double.

In fact float has much more than 7 decimal points. But 7 decimal can be set to desired values. All other are garbage. And converting to longer type you're just moving that garbage into precision decimal points. That's all the magic.

Although converting from one type to another will give you closest possible value. I didn't checked but it could bring much more undefined situations. For example when you expect something like 0.2 (0.20000000125...) you will get 0.199999999999999985.... And it could be closer to 0.2 than previous value. But that is only my assumption.

Floating point are very interesting to work with. I can tell about other magic. If you'll try to get difference of almost equal values, you will get significant error. And this value would be almost useless. The same error could be if you'll divide giant number upon very small number. Although sum of small numbers in 1000+ iterations will give the same useless result because of significant error came from summing operation. All those cases had to be worked out to get good results.

@brarcher
Copy link
Contributor

Having so many macros for different floating point comparisons looks rather cumbersome. Let me ask a few more questions to see if there is no other way.

The macros first store the two values being compared, then perform the computation with the local variables. Because of this, a type must be used. I do not see how this is a hard requirement. What would happen if, in the example of _ck_assert_floating_op_tol, the following was tried:

int _result = (X - Y) OP T * (D);
ck_assert_msg(_result, ...

Could this avoid knowing the exact type, leaving it up to whatever the caller specified them as?

And converting to longer type you're just moving that garbage into precision decimal points. That's all the magic.
Although converting from one type to another will give you closest possible value. I didn't checked but it could bring much more undefined situations.

Consider if one cast a float to a long double. If the original value only had a precision of 7.25 decimal places, one would expect that the long double value to have no more precision than that. The remaining digits would be expected to be noise. A user should not expect to get any additional precision by casting up. The case where it would be troublesome is if casting up resulted in a loss of precision.

@andr-dots
Copy link
Contributor Author

andr-dots commented Nov 30, 2016

What would happen if, in the example of _ck_assert_floating_op_tol, the following was tried:

It was the first suggestion I tried to implement. But I had to reject this idea because I cannot get type of variables (in C99) to apply appropriate modifiers to fprintf (i.e. %f, %lf, %Lf). The only way I can get type of macro argument is the sizeof. But sizeof(int) == sizeof(float). So it could be segmentation fault in some cases. In other it would be type mismatch and wrong results.

@andr-dots
Copy link
Contributor Author

andr-dots commented Nov 30, 2016

Consider if one cast a float to a long double. If the original value only had a precision of 7.25 decimal places, one would expect that the long double value to have no more precision than that. The remaining digits would be expected to be noise. A user should not expect to get any additional precision by casting up. The case where it would be troublesome is if casting up resulted in a loss of precision.

I think is a very bad idea. User could pass first argument as float and second argument as double. But promoting one to another will produce new digits in precision zone. The are not the same. If a macro if only for float numbers user would not be misunderstood. he already knows that float numbers must be passed.

I even had an idea to check the size of arguments. But if one would pass int or long it wouldn't work.

UPDATE: I thought about code size. If we generate three files from one .h.in file, it would decrease code amount. But I do not know autotools at the moment so I'll need some help in this case.

@andr-dots
Copy link
Contributor Author

andr-dots commented Dec 1, 2016

One more example:

    double d = 0.1;
    float f = 0.1;
    float fl = 0.1l; // in fact it would be 0.1f
    printf("%.15lf; %.15f; %.15f\n", d, f, fl);
// 0.100000000000000; 0.100000001490116; 0.100000001490116

So in case of typed macros, if constants passed to a macro they would be converted to a variable type. Thus it resolves the problem when user passes 0.1 comparing it to float variable. It would be just converted to 0.1f. But not promoted. So they would be equal.
Here is the explanation:
http://stackoverflow.com/questions/36840173/precision-loss-from-float-to-double-and-from-double-to-float

@brarcher
Copy link
Contributor

brarcher commented Dec 3, 2016

(Reasoning about floating point math is not easy...)

The link you mentioned, namely,

http://stackoverflow.com/questions/36840173/precision-loss-from-float-to-double-and-from-double-to-float

mentions that promoting a float->{double, long double} or double->{long double} does not change the value. If this is the case, does that not imply that the following:

In fact float has much more than 7 decimal points. But 7 decimal can be set to desired values. All other are garbage. And converting to longer type you're just moving that garbage into precision decimal points. That's all the magic.

is not entirely accurate, as the lower order bits of the significand field would be '0'. The following example may show this:

    volatile float x = 0.1f;
    volatile double y = 0.1;
    printf("%.15lg; %.15lg; %.15g;\n", (double) x, y, x);
    // Prints: 0.100000001490116; 0.1; 0.100000001490116;

That is, the float variable 'x' when cast to a double and printed, and printed directly, is the same value.

Consider if one cast a float to a long double.

I think is a very bad idea. User could pass first argument as float and second argument as double. But promoting one to another will produce new digits in precision zone. The are not the same.

True, but my understanding is that the new digits would be '0' and would not change the value of the data. Thus, if a user did decide to compare a float and a double, but the double happen to exactly represent the same data at the float, then the comparison would still be valid. If, however, they were not expected to be exact, then the tolerance-based macros would be appropriate, no?

I'm still leaning on the side of having one macro that casts to long double then compares. Using separate macros may be pedantic, but it may not be strictly necessary. Do you disagree?

UPDATE: I thought about code size. If we generate three files from one .h.in file, it would decrease code amount.

I'm not too concerned about the size of check.h. If a user does not use a macro in a given file then it should not impact their code size.

@andr-dots
Copy link
Contributor Author

andr-dots commented Dec 4, 2016

That is, the float variable 'x' when cast to a double and printed, and printed directly, is the same value.

Well, yes, they are the same for computer but it still may confuse a programmer who is trying to solve simple task and doesn't know what floating point is, because in decimal representation they are not the same. And it's true only for promotions. Example:

	volatile float x = 0.1;
	volatile double y = 0.1;
	fprintf(stderr, "%.15f %.15lf %d\n", x, y, (x == y));
//0.100000001490116 0.100000000000000 0

I'm not too concerned about the size of check.h. If a user does not use a macro in a given file then it should not impact their code size.

I mean the code size to maintain.

I'm still leaning on the side of having one macro that casts to long double then compares. Using separate macros may be pedantic, but it may not be strictly necessary. Do you disagree?

If I thought if isn't necessary I wouldn't rewrite all the code once again. The example is in the tests. I added one more test to show what I mean (test_ck_assert_ldouble_eq_with_conv).

@andr-dots
Copy link
Contributor Author

andr-dots commented Dec 4, 2016

And in any case float, double and long double separate macros are just give users a chance to choose. One could use ldouble macros for all types. Another does not need double and long double type. So he can use float macros. And float macros should be much faster than double (or ldouble) macros in i586 architecture. For example my tests for only one library already run for about a second. And it's just beginning.

@andr-dots andr-dots force-pushed the master branch 7 times, most recently from f9af9ba to 395f9fd Compare December 8, 2016 05:16
Dotsenko Andrey added 4 commits December 8, 2016 20:33
Each author with his personal info must be on one line (his name, email, etc) and his contribution must be on the next line.
@andr-dots
Copy link
Contributor Author

I've added tests for expressions.

@brarcher
Copy link
Contributor

brarcher commented Dec 9, 2016

The example is in the tests. I added one more test to show what I mean (test_ck_assert_ldouble_eq_with_conv).

float x = 1.1;
long double y = x;
ck_assert_ldouble_eq(x, y);
record_failure_line_num(__LINE__);
ck_assert_ldouble_eq(x, 1.1);

I think that I understand now. At first glance one would have expected that to pass. But, the 1.1 being passed to the macro was originally a long double, were as the float was promoted, means that unless 1.1 can be exactly represented as a float then the comparison will fail. Tricky.

Reviewing the changes one more time, I am comfortable with these additions. Thanks for the effort in adding them, and thanks for the discussion. I'll update the branch, and after the checks complete I'll merge the changes in.

The last piece necessary for this work is adding platform dependent checks for NaN, right? Any thoughts on how best to achieve that in a maintainable way?

@brarcher brarcher merged commit 8b1bccf into libcheck:master Dec 10, 2016
@andr-dots
Copy link
Contributor Author

The last piece necessary for this work is adding platform dependent checks for NaN, right? Any thoughts on how best to achieve that in a maintainable way?

I'll add regular expressions to the master file. I think it can be useful when adding new API. So NaN would be just "-?nan" in the text. Another way, I could add a check to the macro and print always nan as text. But I am not sure about second solution because I do not know what user expects to see. If he always see -nan while debugging and tests would show +nan, it could confuse a little. But it's a much easier way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants