shimh Posted November 26, 2008 Share Posted November 26, 2008 When I run the following code, the result does not look correct. global x x= 0x80000001 ? Format("0x%08X",test(x)) function test(in) return in The expected result is 0x80000001, but we get 0x7FFFFFFF. Actually we always get 0x7FFFFFFF when x is larger than or equal to 0x80000000. Now we change the code to the following, which is supposed to be equivalent: global x x= 0x1 ? Format("0x%08X",test(x)) function test(in) in += 0x80000000 return in We get 0x80000000 this time, which can be larger than the previous ceiling 0x7FFFFFFF. However, the value is not correct. Then we change the code again to the following: global x x= 0x1 ? Format("0x%08X",test(x)) function test(in) in += 2147483648 return in And we get the expected 0x80000001. One thing is that when we run ?0x80000000, and we get 2147483647. However, if we convert it in http://www.parkenet.com/apl/HexDecConverter.html, we get 2147483648. We suppose the latter is the proper result, as we see in the third code example. Is it a bug or something? Thanks Link to comment Share on other sites More sharing options...
AzeoTech Posted November 26, 2008 Share Posted November 26, 2008 DAQFactory uses 32 signed integers when doing the hex conversions. This means you only get 31 bits of precision with 1 bit used to determine sign. This means that the maximum it can convert is 0x7fffffff. This is talked about in the manual and in this post: (post link) Really if you try and do anything integer related (ie. bit functions) with values > 0x7fffffff you will get unexpected results. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.