Testing Floating Point Numbers Accuracy

This should be a no brainier for anyone who has been coding for awhile but computer math can be funky at times.  To demonstrate how accuracte a floating point number is I opened up my python interperter and ran this mathimatical gem.

>>> float(1/998001)

As you can see python gives up pretty quickly calculating this number. Dividing 1 by 998001 will create a decimal sequence of every three digit number in order.

Now lets take a look a python’s decimal data type.

from decimal import Decimal
>>> Decimal(1/998001)

Ok, the decimal length is bigger but it sequence turns to garbage at around the same point where the floating point number gave up. There are a few ways to improve your programming math but the given answer will still be bound by the constants of being computed by a computer.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s