Login | Register   
LinkedIn
Google+
Twitter
RSS Feed
Download our iPhone app
TODAY'S HEADLINES  |   ARTICLE ARCHIVE  |   FORUMS  |   TIP BANK
Browse DevX
Sign up for e-mail newsletters from DevX


Tip of the Day
Language: Visual Basic
Expertise: Beginner
May 11, 1999

Compare Floating Point Values Using the Round Function

When you have to compare the results of floating point expressions, you can't rely on the "=" operator due to the finite precision of Single or Double variables. To see this concept demonstrated, use this code:
 
Dim i As Integer, d As Double
For i = 1 To 10
	d = d + CDbl(0.1)
Next
MsgBox (d = 1)	' displays "False"
To more easily compare two floating numbers, use the new VB6 Round function, which rounds a number to the desired number of decimal digits. For example, you can rewrite the previous test like this:
 
' the difference is less than 1E-12
MsgBox (Round(d, 12) = 1)  ' displays "True"
You can also use this method to check whether A and B variables contain values that match up to their 12th decimal digit:
 
If Round(A - B, 12) = 0 Then Print "Equal"
Francesco Balena
 
Comment and Contribute

 

 

 

 

 


(Maximum characters: 1200). You have 1200 characters left.

 

 

Sitemap
Thanks for your registration, follow us on our social networks to keep up-to-date