-
Notifications
You must be signed in to change notification settings - Fork 219
Add comparison macros for floating point numbers (ck_assert_floating_*) #69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I'm a newbye to github. Could somebody explain why does AppVeyor fails and how to debug this? |
It seems that MSVC has build-time check for this. |
76c3ad5
to
b9d11c7
Compare
Need some help again. I've figured out how to open logs for ms builds. With nan I'll deal somehow on different compilers. But logs for platform=ms are unclear for me. Can I get tests log somehow with asserts text? |
I've removed nan, finite and infinite asserts until get managed to check them on any compiler (ignoring nan's sign). |
Jenkins: ok to test |
* | ||
* @since 0.10.1 | ||
*/ | ||
#define ck_assert_floating_eq(X, Y) _ck_assert_floating(X, ==, Y) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may not want to offer a way to use the == operator on two floating point numbers. Because floating point math is not associative and may follow different rounding models, the outcome of a computation may not be exactly what one expects. The ck_assert_floating_eq_prec() call below is a much better option. I would request that this and the ck_assert_floating_ne() methods be removed.
@@ -104,6 +104,22 @@ static master_test_t master_tests[] = { | |||
{ "Simple Tests", CK_FAILURE, "Assertion 'x >= y' failed: x == 2, y == 3" }, | |||
{ "Simple Tests", CK_FAILURE, "Assertion '1%d >= 3%f' failed: 1%d == 0, 3%f == 1" }, | |||
{ "Simple Tests", CK_PASS, "Passed" }, | |||
/* Tests on floating macros */ | |||
{ "Simple Tests", CK_FAILURE, "Assertion 'x == y' failed: x == 1.100000, y == 1.200000" }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You will need to rebase against master. I added a change today which will add an entry to this table. Now one will need to include the name of the method under test. For example, the first entry will now become:
{ "Simple Tests", "test_ck_assert_floating_eq", CK_FAILURE, "Assertion 'x == y' failed: x == 1.100000, y == 1.200000" }
do { \ | ||
long double _ck_x = (X); \ | ||
long double _ck_y = (Y); \ | ||
unsigned char _prec = (PREC); \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PREC parameter currently assumes that one wants to compare the precision of decimal places assuming a power of 1. However, it may be likely that one is interested in comparing using other powers. For example, if I have two floating point numbers: 1e10 and 1.1e10 I may be interested in checking that they are within 1e8 of each other.
Consider changing this to allow one to pass in the maximum value that the two numbers can be apart and still be considered 'equal'.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Power of 10 is often used in calculations when assuming fix point calculations which are used in embedded systems. Through fixed point types are unable under C standard using float is much or less a good way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A'll consider adding another function for eps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the explanation. I now have a better appreciation for where you are trying to go with this macro.
Given your use of the macro, having this to help with math which is expected to be an approximate for fixed point seems reasonable. I still believe that for the floating point macros to be complete a more general macro which allow one to pass the threshold should be provided. Can you provide both in your change?
} | ||
END_TEST | ||
|
||
START_TEST(test_ck_assert_floating_eq_prec) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When testing this method should data available in cfloat.h be used to determine the correctness? This will result in more extreme examples that check boundary conditions.
I'm OK considering this change. However, I warn that working with floating point values is hard and determining good and meaningful comparisons is non-trivial. Can you share your expected use case for these macros and the application you are considering using them to unit test? |
We use those tests in our commercial project so I cannot give full example. But it looks like this: |
#define _ck_assert_floating(X, OP, Y) do { \ | ||
long double _ck_x = (X); \ | ||
long double _ck_y = (Y); \ | ||
ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: %s == %.6Lf, %s == %.6Lf", #X" "#OP" "#Y, #X, _ck_x, #Y, _ck_y); \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having no expectation on the values being compared using the format specifier %.6Lf may be a little restrictive. For example, if X were 0.000000012 then the value of 0.000000 would be printed. However, the format %Le would print 1.200000e-08.
Would you be opposed to changing the format specifier to be more general?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about it. But tests should give human readable output. So in case of 0.001 and 1.0e-3, 0.001 is clear. But 1.0e-3 spends your time to do some calculations in your mind. In this case %Lg is universal solution. Just didn't checked what output could be on different compilers and system. In any case I'll make some changes in a week or two to tests making use regular expressions for output analysis (if don't mind) through nan and -nan problem appeared, so no problem, I'll add %Lg everywhere. I can contribute to open source only in free from work time so for now it wouldn't be fast.
Turns out that the Visual Studios build was never printing the output of the unit tests, even on failure. This is now resolved in master (this pull request). The Visual Studios build will now always emit the unit test output. Thanks for pointing this out. |
1c1b0c1
to
0b62863
Compare
long double _ck_y = (Y); \ | ||
int _ck_p = (PREC); \ | ||
int _prec = -_ck_p; \ | ||
ck_assert_msg(_prec <= 0, \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this assertion is to help protect the usage of this macro consider changing its check to something that relates directly to the user's input. That is,
ck_assert_msg((PREC) > 0, ...)
This will help a user make sense of the failure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
User see custom message after condition and condition (PREC) > 0 won't be seen. How could it help?
There are a few failures in the unit tests. One of them I neglected to warn about, sorry about that. In addition to adding the name of the unit test to the master tests table, one will need to add the following call to the unit tests as well:
The second failure is here: It appears that the test_ck_assert_floating_eq_prec_min_prec unit test failed (as expected) but it did not fail in the way it was expected to fail. |
My main problem is that |
There is a way to disable all the timeout tests:
With that you will find that unit tests run very quickly. |
df857d5
to
2c445e8
Compare
Well I've done some research. %Lg with default precision is confusing through 0.1 is not 0.1 because of digits at the end of number. So made macro CK_FLOATING_DIG that can be set by user to determine his own precision (for example DECIMAL_DIG for long double from C99). In documentation I wrote restriction on use of floating point numbers. And I cannot test bounds of precision because of different representations on x32 and x64 systems (or on different compilers/platforms, didn't dig for the reason). And because of big error coming from binary representation. But tests are useful in any case. Macros for threshold and macros for eps would be added some later through for now I have no need of them in the project I work for. Eps would be next stage in experiment of improving precision of algorithms. |
It's more complicated. For example, numbers 0.1f, 0.1 and 0.1l are not equal. So if someone would test float variable but second argument would be of double type (just number) the result would be erroneous. So to prevent situations when user doesn't understand what is happening I added macros for every type. Moreover if we would print a float number converted to long double type, than we would print different result than user expects to see. |
This would apply for converting from a type with more precision (long double) to one with less precision (float). However, if one always converts to long double how would this result in an issue? Is there evidence that converting from float to long double results in 'garbage' (i.e. undefined behavior)? |
Here is the example:
printf("%g; %lg; %d\n", 0.1f, 0.1, 0.1f == 0.1);
// 0.1; 0.1; 0
// As shows printf they are equal, but...
printf("%.15g; %.15lg; %d\n", 0.1f, 0.1, 0.1f == 0.1);
// 0.100000001490116; 0.1; 0
volatile float x = 0.1f;
volatile double y = 0.1;
printf("%.15lg; %.15lg; %d\n", (double) x, y, ((double) x) == y);
// 0.100000001490116; 0.1; 0
// That was happening when converting float to double. In fact float has much more than 7 decimal points. But 7 decimal can be set to desired values. All other are garbage. And converting to longer type you're just moving that garbage into precision decimal points. That's all the magic. Although converting from one type to another will give you closest possible value. I didn't checked but it could bring much more undefined situations. For example when you expect something like 0.2 (0.20000000125...) you will get 0.199999999999999985.... And it could be closer to 0.2 than previous value. But that is only my assumption. Floating point are very interesting to work with. I can tell about other magic. If you'll try to get difference of almost equal values, you will get significant error. And this value would be almost useless. The same error could be if you'll divide giant number upon very small number. Although sum of small numbers in 1000+ iterations will give the same useless result because of significant error came from summing operation. All those cases had to be worked out to get good results. |
Having so many macros for different floating point comparisons looks rather cumbersome. Let me ask a few more questions to see if there is no other way. The macros first store the two values being compared, then perform the computation with the local variables. Because of this, a type must be used. I do not see how this is a hard requirement. What would happen if, in the example of _ck_assert_floating_op_tol, the following was tried:
Could this avoid knowing the exact type, leaving it up to whatever the caller specified them as?
Consider if one cast a float to a long double. If the original value only had a precision of 7.25 decimal places, one would expect that the long double value to have no more precision than that. The remaining digits would be expected to be noise. A user should not expect to get any additional precision by casting up. The case where it would be troublesome is if casting up resulted in a loss of precision. |
It was the first suggestion I tried to implement. But I had to reject this idea because I cannot get type of variables (in C99) to apply appropriate modifiers to |
I think is a very bad idea. User could pass first argument as float and second argument as double. But promoting one to another will produce new digits in precision zone. The are not the same. If a macro if only for float numbers user would not be misunderstood. he already knows that float numbers must be passed. I even had an idea to check the size of arguments. But if one would pass UPDATE: I thought about code size. If we generate three files from one .h.in file, it would decrease code amount. But I do not know autotools at the moment so I'll need some help in this case. |
One more example: double d = 0.1;
float f = 0.1;
float fl = 0.1l; // in fact it would be 0.1f
printf("%.15lf; %.15f; %.15f\n", d, f, fl);
// 0.100000000000000; 0.100000001490116; 0.100000001490116 So in case of typed macros, if constants passed to a macro they would be converted to a variable type. Thus it resolves the problem when user passes 0.1 comparing it to float variable. It would be just converted to 0.1f. But not promoted. So they would be equal. |
(Reasoning about floating point math is not easy...) The link you mentioned, namely, mentions that promoting a float->{double, long double} or double->{long double} does not change the value. If this is the case, does that not imply that the following:
is not entirely accurate, as the lower order bits of the significand field would be '0'. The following example may show this:
That is, the float variable 'x' when cast to a double and printed, and printed directly, is the same value.
True, but my understanding is that the new digits would be '0' and would not change the value of the data. Thus, if a user did decide to compare a float and a double, but the double happen to exactly represent the same data at the float, then the comparison would still be valid. If, however, they were not expected to be exact, then the tolerance-based macros would be appropriate, no? I'm still leaning on the side of having one macro that casts to long double then compares. Using separate macros may be pedantic, but it may not be strictly necessary. Do you disagree?
I'm not too concerned about the size of check.h. If a user does not use a macro in a given file then it should not impact their code size. |
Well, yes, they are the same for computer but it still may confuse a programmer who is trying to solve simple task and doesn't know what floating point is, because in decimal representation they are not the same. And it's true only for promotions. Example: volatile float x = 0.1;
volatile double y = 0.1;
fprintf(stderr, "%.15f %.15lf %d\n", x, y, (x == y));
//0.100000001490116 0.100000000000000 0
I mean the code size to maintain.
If I thought if isn't necessary I wouldn't rewrite all the code once again. The example is in the tests. I added one more test to show what I mean (test_ck_assert_ldouble_eq_with_conv). |
And in any case float, double and long double separate macros are just give users a chance to choose. One could use ldouble macros for all types. Another does not need double and long double type. So he can use float macros. And float macros should be much faster than double (or ldouble) macros in i586 architecture. For example my tests for only one library already run for about a second. And it's just beginning. |
f9af9ba
to
395f9fd
Compare
…ssert_floating_*)
Each author with his personal info must be on one line (his name, email, etc) and his contribution must be on the next line.
I've added tests for expressions. |
I think that I understand now. At first glance one would have expected that to pass. But, the 1.1 being passed to the macro was originally a long double, were as the float was promoted, means that unless 1.1 can be exactly represented as a float then the comparison will fail. Tricky. Reviewing the changes one more time, I am comfortable with these additions. Thanks for the effort in adding them, and thanks for the discussion. I'll update the branch, and after the checks complete I'll merge the changes in. The last piece necessary for this work is adding platform dependent checks for NaN, right? Any thoughts on how best to achieve that in a maintainable way? |
I'll add regular expressions to the master file. I think it can be useful when adding new API. So NaN would be just "-?nan" in the text. Another way, I could add a check to the macro and print always nan as text. But I am not sure about second solution because I do not know what user expects to see. If he always see -nan while debugging and tests would show +nan, it could confuse a little. But it's a much easier way. |
Those macros could help testing geometry functions with specified precision.