Odd issue comparing two numbers

[Chosenatrandom]

Hi - I’m converting an algorithm from mat lab, and it’s going well, but I’m getting an odd error. When I print the output with ?n[100], I get 0.054, but when I try and compare the value to 0.054, I don’t get the correct evaluation. A working example follows:

global n

for (private count = 0.004, count <= 4, count += 1/2000)

   n.append(count)

endfor

? n[100] // output is 0.054

// now here is the problem

if (n[100] == 0.054)

   ? “Success”

endif

Thanks for your help

 

[AzeoTech]

OK, this is actually a feature of floating point numbers in computers and has nothing to do specifically with DAQFactory.  The short of it is that while computers store integers exactly, the decimal part (right of the decimal place) is almost never stored exactly due to the way floating point values work.  DAQFactory uses double precision floating point so the effect is smaller than single precision, but it still means that 0.1, for example, is probably stored as 0.0999999999999998 or something like that.  When DAQFactory goes to display the value it shows as 0.1 because of various algorithms to clean up floating values, but internally it is actually slightly off from 0.1.  You actually rarely see this problem because the system does some automatic rounding, except for when you do something like what you did where you added a bunch of fractional values together and then compared it to a value that was directly entered.  The slight offset of 1/2000 is getting compounded so that by the time it is 0.054 it is actually slightly different.  Alas, there is no real easy way to see this.  However, try replacing your ? n[100] with:

? n[100]// output is 0.054
? n[100] == 0.054
? n[100] < 0.054001
? n[100] > 0.053999
? floor(n[100] * 1000) / 1000 == 0.054

What you'll see is:

0.054
0
1
1
1

The 0 is the comparison you did and doesn't compute because n[100] is actually slightly off from 0.054.  The next two show that it is just slightly off, and the last one shows how you can simply truncate anything after the 3rd decimal place and do the comparison and it works.

[Chosenatrandom]

Many thanks - I’ll put this to good use today.