devxlogo

Compare Floating Point Values Using the Round Function

Compare Floating Point Values Using the Round Function

When you have to compare the results of floating point expressions, you can’t rely on the “=” operator due to the finite precision of Single or Double variables. To see this concept demonstrated, use this code:

 Dim i As Integer, d As DoubleFor i = 1 To 10	d = d + CDbl(0.1)NextMsgBox (d = 1)	' displays "False"

To more easily compare two floating numbers, use the new VB6 Round function, which rounds a number to the desired number of decimal digits. For example, you can rewrite the previous test like this:

 ' the difference is less than 1E-12MsgBox (Round(d, 12) = 1)  ' displays "True"

You can also use this method to check whether A and B variables contain values that match up to their 12th decimal digit:

 If Round(A - B, 12) = 0 Then Print "Equal"
devxblackblue

About Our Editorial Process

At DevX, we’re dedicated to tech entrepreneurship. Our team closely follows industry shifts, new products, AI breakthroughs, technology trends, and funding announcements. Articles undergo thorough editing to ensure accuracy and clarity, reflecting DevX’s style and supporting entrepreneurs in the tech sphere.

See our full editorial policy.

About Our Journalist