InterviewSolution
Saved Bookmarks
InterviewSolution
| 1. |
Solve : VB6 number formatting? |
|
Answer» If I run the following code: By definition real numbers do not have exact values. When you count by 0.5 you are not saying exactly 0.5, but a number very close to it. So by that definition 0.5 times 4 is not exactly 2, but a value very close to two, but it can not be two as a integer number. I suppose I know what you're getting at, but this is nonsense. Some non-integer real numbers can be represented exactly in a number base, some cannot. 0.5 is a rational number which can be represented exactly in base 10, but not in BINARY (hence the problem). The OP needs to read a good programming book, with particular reference to data types, binary representation, floating point arithmetic, and rounding errors. Salmon Trout, My remark was in the context of Microsoft basic. The NOTATION "0.5" could indicate a value result of a division of an integer. But it might be a real number. In Microsoft Basic it will be cast as a real number and it no longer sis the result of a division of an integer. Unless the programmer did something to force it to another type. There are formats used in MS Basic the keep numbers in a form that is related to an integer. The currency type is such. It is actually a long integer. But when printed or displayed, it is shown as a decimal fraction. Once a number as been converted to a real floating point number, it loses its virginity forever. It no longer is an exact value. By definition real numbers are not exact values, even if they are. In other words, a real number has a component that is not quantized. If the number is quantized, it could be a fixed-point number in either binary or an hybrid number system. Anyway, he can resolve the issue by user either INT(), which truncates, or ROUND(), which rounds off numbers to a given number of decimal places. Code: [Select]If i = 2 Then MsgBox "bla"Could be rewritten as: Code: [Select]If int(i+0.1) = 2 Then MsgBox "bla"Good night. Quote from: Linux711 on March 18, 2012, 10:07:38 PM If I run the following code: Code: [Select]Dim i As Integer Dim j As Double for i = 0 to 5000 j = CDbl(i) / 1000# if j = 2 Then MsgBox "Blah" Next P.S: it's never exactly 2 in the original because 0.001 cannot be accurately represented using floating point. This version attempts to avoid that problem by not incrementing using a INEXACT floating point number. Instead, each iteration increments using an integral value, and that value is used in a division. This is pretty much a scaled integer approach. The problem you were seeing was because of the accumulated imprecision from adding an inprecise floating point representation to itself. Here's another version that might work, but it will probably be slower, because it uses the Decimal subtype of Variant. (It's not so much slow because of the Variant type but because all the Decimal operations are dealt with through software) Code: [Select]Dim i As Variant i=CDec(0) for i = 0 to 5 step CDec(0.001) If i = 2 Then MsgBox "bla" Next i Quote Once a number as been converted to a real floating point number, it loses its virginity forever. It no longer is an exact value.As Salmon Trout just said, some numbers can be represented perfectly in floating point. Most whole numbers are simple to represent perfectly in floating point, as long as the number doesn't require a exponent to represent. Any number that fits in the mantissa will be a perfect representation of that value. The issue here is not that a single floating point value was inaccurate; the Visual Basic runtime works with this- 0.1*2 will equal 0.2 even though the actual values are not equal, because VB intrinsically uses a tolerance value. The problem is in the use of floating point numbers that cannot be accurately represented as accumulators; 0.001, for example- because this compounds the ERROR to the point where the accumulated error becomes significant. Running my older Expression Evaluator (written in VB6) on the expression SEQ(X,X,1,5,0.01) (which creates a sequence of values from 1 to 5 in steps of 0.01) there is no visible error in the output until 1.6, which is output as 1.69999999998 (or something similar) That is the point at which the floating point error "escapes" the built in tolerances of the floating point library in use. Of course, even so, comparing two floating point numbers for equality is typically not a good idea because you cannot be sure of the processes that generated those numbers. Usually, the test could merely take the difference and assert that it is smaller than a given tolerance: Code: [Select]Const tolerance As Double-0.001 Public Function TestEqual(ByRef Value1 As Double,ByRef Value2 As Double) As Boolean TestEqual = (Abs(Value1-Value2) < tolerance) End Function |
|