Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
DiscussionsAccessExcelInfoPathOutlookPowerPointPublisherWord
DirectoryUser Groups
Related Topics
Outlook ExpressInternet ExplorerWindowsMS Server ProductsMore Topics ...

MS Office Forum / Excel / Excel Errors / July 2007

Tip: Looking for answers? Try searching our database.

Calculations

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Tim O'Connor - 31 Jul 2007 14:00 GMT
I rounded several numbers to two decimals then copies and pasted the values,
not the function.  When I sum the numbers, the answer should be zero but but
the sum is -2.27373675443232E-13.  I thought rounding and pasting the values
would get rid of the these types of errors.
Jerry W. Lewis - 31 Jul 2007 22:14 GMT
If no number has more than 2 decimal places and you are only
adding/subtracting, then you can round the final result to 2 decimal places
without violence to the calculation.

Computers work in binary.  In binary, most terminating decimal fractions are
non-terminating binary fractions that can only be approximated (just as 1/3
can only be approximated as a decimal fraction).  Since the inputs are only
approximately what you intended, it should be be no surprise when their sum
is only approximately what you intended.  For example =(0.3-0.2-0.1) will
return -2.8E-17 instead of zero, due to the fact that all three decimal
fractions must be approximated.  The approximations used by Excel (and almost
all other general purpose software) follow the IEEE 754 standard for double
precision.

Jerry

> I rounde several numbers to two decimals then copies and pasted the values,
> not the function.  When I sum the numbers, the answer should be zero but but
> the sum is -2.27373675443232E-13.  I thought rounding and pasting the values
> would get rid of the these types of errors.
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.